{"id":15062846,"url":"https://github.com/tinkershack/muxfarm","last_synced_at":"2026-01-02T14:34:40.718Z","repository":{"id":235458546,"uuid":"662667036","full_name":"tinkershack/muxfarm","owner":"tinkershack","description":"Transmux, transcode and package media files from local and remote sources","archived":false,"fork":false,"pushed_at":"2024-04-23T08:29:00.000Z","size":19,"stargazers_count":0,"open_issues_count":5,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2024-04-23T13:24:52.766Z","etag":null,"topics":["ffmpeg","transcoding","transmux"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tinkershack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-07-05T16:09:23.000Z","updated_at":"2024-04-23T13:24:58.174Z","dependencies_parsed_at":"2024-04-26T03:15:08.342Z","dependency_job_id":null,"html_url":"https://github.com/tinkershack/muxfarm","commit_stats":null,"previous_names":["tinkershack/muxfarm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinkershack%2Fmuxfarm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinkershack%2Fmuxfarm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinkershack%2Fmuxfarm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tinkershack%2Fmuxfarm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tinkershack","download_url":"https://codeload.github.com/tinkershack/muxfarm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243741301,"owners_count":20340443,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ffmpeg","transcoding","transmux"],"created_at":"2024-09-24T23:47:27.416Z","updated_at":"2026-01-02T14:34:40.713Z","avatar_url":"https://github.com/tinkershack.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MuxFarm\n\nA distributed media processing system for transmuxing, transcoding, and packaging media files from various sources into streaming-ready formats.\n\n## Overview\n\nMuxFarm provides a scalable architecture for processing media content, supporting ingestion from multiple storage backends and conversion to modern streaming formats like MPEG-DASH and HLS. Built with Go and leveraging FFmpeg for media operations, it's designed to handle large-scale media workflows.\n\n## Key Features\n\n### Media Processing\n\n- **Transmuxing**: Container format conversion (MP4, WebM, FLV, MPEG)\n- **Transcoding**: Video (H.264, VP9, AV1) and audio (AAC, MP3, Opus) codec support\n- **Adaptive Streaming**: MPEG-DASH and HLS packaging with segmentation\n- **Quality Control**: Bitrate and resolution management\n\n### FFmpeg Integration\n\nMuxFarm leverages FFmpeg's powerful multimedia framework through a comprehensive Go wrapper:\n\n- **ffprobe**: Media analysis and metadata extraction\n  - Container format detection\n  - Stream information parsing\n  - Duration and bitrate analysis\n  - Codec identification\n\n- **ffmpeg**: Core media processing engine\n  - Stream mapping and filtering\n  - Real-time transcoding with hardware acceleration support\n  - Custom filter graphs for complex transformations\n  - Pipe-based I/O for memory-efficient processing\n\n- **Supported FFmpeg Features**:\n  - Multiple input/output streams\n  - Format-specific optimization flags\n  - Metadata preservation and manipulation\n  - Time-based seeking and trimming\n  - Custom encoding parameters\n\n### Multi-Source Ingestion\n\n- Local filesystem (LFS)\n- HTTP/HTTPS URLs\n- AWS S3 buckets\n- Google Cloud Storage\n- Git repositories\n\n### Distributed Architecture\n\n- gRPC-based service communication\n- MongoDB for job persistence and state tracking\n- Redis for distributed coordination and locking\n- Horizontal scaling of processing workers\n\n## Architecture\n\n```text\n┌─────────────────┐    ┌─────────────────┐\n│   Client App    │───▶│   MIMO Server   │\n└─────────────────┘    └─────────────────┘\n                                │\n                       ┌─────────────────┐\n                       │   Mops Service  │\n                       └─────────────────┘\n                                │\n                       ┌─────────────────┐\n                       │    MongoDB      │\n                       └─────────────────┘\n                                │\n        ┌───────────────────────┼───────────────────────┐\n        ▼                       ▼                       ▼\n┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐\n│ IngestSplitter  │    │   AtomRacer     │    │   AtomPuller    │\n└─────────────────┘    └─────────────────┘    └─────────────────┘\n                                │\n                       ┌─────────────────┐\n                       │ FFmpeg Wrapper  │\n                       │  - ffprobe      │\n                       │  - ffmpeg       │  \n                       └─────────────────┘\n```\n\n### Core Components\n\n- **MIMO Server**: gRPC API server handling client requests\n- **Mops Service**: Media operations service managing job lifecycle  \n- **IngestSplitter**: Breaks ingestion requests into atomic processing tasks\n- **AtomRacer**: Coordinates and monitors atom processing workers\n- **AtomPuller**: Downloads media from remote sources to local storage\n\n## Getting Started\n\n### Prerequisites\n\n- Go 1.19+\n- FFmpeg (for media processing)\n- MongoDB (for job persistence)\n- Redis (for distributed coordination)\n\n### Configuration\n\nCreate a `muxfarm.yaml` configuration file:\n\n```yaml\nMuxfarm:\n  MIMO:\n    HostName: \"localhost\"\n    Port: \"50050\"\n\nDocumentStore: \"mongodb\"\nMongoDB:\n  Name: \"mongodb\"\n  URI: \"mongodb://localhost:27017\"\n  DBName: \"muxdb\"\nDLMRedis:\n  Name: \"redis\" \n  URI: \"localhost:6379\"\n  DBNumber: 11\n```\n\n### Running the System\n\n1. Start the MIMO server:\n\n```bash\n./muxfarm mimo\n```\n\n2. Start processing workers:\n\n```bash\n./muxfarm stitch ingestSplitter\n./muxfarm stitch atomRacer\n```\n\n## API Usage\n\nMuxFarm exposes a gRPC API for submitting media processing jobs:\n\n```go\n// Submit a media processing job\nmediain := \u0026plumber.MediaIn{\n    Input: []*plumber.Media{\n        {\n            Storagetype: plumber.StorageType_STORAGE_HTTP,\n            Uri: \"https://example.com/video.mp4\",\n        },\n    },\n    Operation: \u0026plumber.Operation{\n        // Define processing operations\n    },\n}\n\n// Returns a job ID for tracking\njobID, err := client.Ingest(ctx, mediain)\n```\n\n## Output Formats\n\nMuxFarm generates streaming-optimized output:\n\n- **MPEG-DASH**: `.mpd` manifests with segmented media\n- **HLS**: `.m3u8` playlists with transport segments\n- **Fragmented MP4**: Initialization and media segments\n- **Multi-bitrate**: Adaptive streaming support\n\n## Development\n\n### Protocol Buffers\n\nThe system uses Protocol Buffers for service definitions:\n\n- `plumber/plumber.proto`: Core service API\n- `fixtures/fixtures.proto`: Data persistence models\n- `plumber/state.proto`: Job state management\n\nRegenerate protobuf code after changes:\n\n```bash\nmake proto  # or protoc commands\n```\n\n### Adding Storage Backends\n\nImplement the storage interface in `plumber/plumber.go` and add the new type to `StorageType` enum.\n\n## Monitoring\n\nThe system tracks job states through MongoDB collections:\n\n- `ingestCollection`: Top-level ingestion jobs\n- `atomCollection`: Individual processing tasks\n\nJob states progress through: `STATE_UNSPECIFIED` → `PULL_OK` → `INGEST_OK`\n\n## Contributing\n\nMuxFarm is designed to be extensible. Key areas for contribution:\n\n- Additional codec support\n- New storage backend integrations  \n- Enhanced monitoring and observability\n- Performance optimizations\n\n## License\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinkershack%2Fmuxfarm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftinkershack%2Fmuxfarm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftinkershack%2Fmuxfarm/lists"}