{"id":16525769,"url":"https://github.com/vade/ffmpegmediaextension","last_synced_at":"2025-03-16T19:31:14.492Z","repository":{"id":250229357,"uuid":"833845441","full_name":"vade/FFMPEGMediaExtension","owner":"vade","description":"Use macOS Sequia's new Media Extensions to allow playback of FFMPEG supported media via AVFoundation ","archived":false,"fork":false,"pushed_at":"2024-08-09T17:24:26.000Z","size":1380,"stargazers_count":32,"open_issues_count":5,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-18T20:31:02.700Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Objective-C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vade.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-25T22:02:33.000Z","updated_at":"2024-10-15T15:49:13.000Z","dependencies_parsed_at":"2024-08-09T18:51:49.937Z","dependency_job_id":null,"html_url":"https://github.com/vade/FFMPEGMediaExtension","commit_stats":null,"previous_names":["vade/ffmpegmediaextension"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vade%2FFFMPEGMediaExtension","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vade%2FFFMPEGMediaExtension/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vade%2FFFMPEGMediaExtension/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vade%2FFFMPEGMediaExtension/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vade","download_url":"https://codeload.github.com/vade/FFMPEGMediaExtension/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221667211,"owners_count":16860570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T17:06:41.692Z","updated_at":"2025-03-16T19:31:14.486Z","avatar_url":"https://github.com/vade.png","language":"Objective-C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FFMPEGMediaExtension\n\nThis is an attempt to build a macOS 15.0 MediaExtension for libAV* which in theory would allow for native AVFoundation container parsing and decode of FFMPEG enabled Container formats (.mkv, .wmv, .avi etc) and Codecs (ie VP9, VP8, DIVX, XVID etc).\n\nYou can think of Media Extensions as a modern macOS replacement for old macOS 7, 8, 9 and early macOS X Quicktime Extensions.\n\nYou can think of this project as an attempt to make a one size fits all Media Extension like the old Perian software.\n\nSince Media Extensions are shipped in a host app, this has also provides a Host app.\n\nWe have basic frame decoding working! \n\u003cimg width=\"1012\" alt=\"image\" src=\"https://github.com/user-attachments/assets/90c76ba3-4fa9-40df-869a-a9c265afb21d\"\u003e\n\nBut there is a ton left to do!\n- [ ] Asset / Track Metadata\n- [ ] GOP Support\n- [ ] Additional Codecs\n\n## Goal\n\nProvide additional media format playback support for any macOS app which leverages AVFoundation / Core Media as its underlying playback / decode engine\n\n* Finder\n* Quicklook\n* Quicktime Player\n* Any other app which opts in by calling:\n    * `MTRegisterProfessionalVideoWorkflowFormatReaders()` for format parsing support\n    * `VTRegisterProfessionalVideoWorkflowVideoDecoders()` for additional decoder support\n\n\n## Requirements:\n\n* MacOS 15 Beta 4 + \n* XCode 16 Beta 4 + \n\n## Building:\n\nIf you want to build FFMPEG libAVFormat and libAVCodec:\n\n`brew install automake nasm pkg-config`\n\nFFMPEG is installed as a submodule - so ensure you git checkout the submodule and init\n\n`cd ffmpeg`\n```\n./configure --prefix=/usr/local --enable-static --disable-shared --disable-programs --disable-doc --disable-avdevice --disable-avfilter --disable-postproc --disable-swscale --disable-swresample --arch=arm64 --disable-avfoundation\n```\n\n`make -j$(sysctl -n hw.ncpu)`\n\nThis will build you the statically linked libAVFormat.a and libAVCodec.a libries \n\n## Design\n\nMEFormatReader (LibAVFOrmatReader)\n* [x] Plugin instantiation via `LibAVFormatReaderFactory`\n* [x] Register a limited number of supported container formats via plist / exported content types\n    * [x] MKV\n    * Adding new Containers requires editing multiple annoying Plist entries\n* [x] Basic Container parsing via libavformat (`LibAVFormatReader`)\n* [ ] Container metadata to AVMetadataItems\n* [x] Track Parsing (Audio and Video only right now) (`LibAVTrackReader`)\n* [ ] Track specific metadata to AVMEtadataItems\n* [x] CMFormatDescriptions for our Tracks\n* [x] Seek / Stepping via `LibAVSampleCursor`\n* [ ] Provide additional sample metadata that some formats require\n    * [ ] Sample Sync info\n    * [ ] Sample Dependency info\n    * [ ] Post Decode Processing Data \n\nMEVideoDecoder\n* Not yet started\n* This means today we only add container support, not yet additional codecs\n* Stay tuned\n\n## Todo:\n\n* [x] libavformat / libavcodec, libavutil compiled as a .a\n* [x] ffmpeg headers inlcuded\n* [x] Swift Modulemap + Obj-C libav*\n* [ ] Implement necessary Media Extension protocols\n    * [x] MEFormatReaderExtension\n    * [x] MEFormatReader (partial)\n    * [x] METrackReader\n    * [x] METrackInfo\n    * [x] MESampleCursor (partial)\n    * [ ] MEVideoDecoderExtension\n    * [ ] MEVideoDecoder\n    * [ ] MEVideoDecoderPixelBufferManager\n* [x] impelment Extension requirements (plists, entitlements, etc)\n* [ ] Memory Management pass in ObjC / C \n\n## Strategy\n\nWe Eventually require 2 Bundle targets that are Media Extensions.\n\n* A Media Extention that implements `MEFormatReader`, which will allow container parsing via `libavformat` and provide container metadata, track metadata, and compressed sample metadata. \n\n* A Media Extension that impelments `MEVideoDecoder` - a Video Decompressor which will allow for codec decompression via `libavcodec` and provide pixel buffers from appropriate samples. \n\n\nFFMPEGs library more or less follows similar design patterns to AVFoundation. The following isnt totally accurate, but more or less gets the point across:\n\n* `CMTime` \u003c~\u003e `AVRational` \n* `CMMediaType` \u003c~\u003e `AVMediaType`\n* `CMFormatDescription` \u003c~\u003e `AVCodecParameters`\n* `CMSampleBuffer` \u003c~\u003e `AVPacket`\n* `CMBlockBuffer` \u003c~\u003e `AVBuffer`\n* `CVPixelBuffer` \u003c-\u003e `AVFrame` (decoded video)\n\n* `AVAsset` \u003c~\u003e `AVFormatContext`\n* `AVAssetTrack` \u003c~\u003e `AVStream`\n* `AVSampleCursor` \u003c~\u003e `AVIOContext` \n\n\n\n\nLeverage AVIOContext to allow a MEByteSource to driven by a AVFormatContext to get us container parsing and metadata\n\n## Running Notes:\n\n* Weirdly MediaExtension headers imply macOS 14 supported?\n* entitlements for Media Extensions\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvade%2Fffmpegmediaextension","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvade%2Fffmpegmediaextension","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvade%2Fffmpegmediaextension/lists"}