{"id":50509713,"url":"https://github.com/dmatking/m5stack-tab5-video-stream","last_synced_at":"2026-06-02T19:01:12.562Z","repository":{"id":352762181,"uuid":"1215527317","full_name":"dmatking/m5stack-tab5-video-stream","owner":"dmatking","description":"MJPEG video + synchronized audio streaming to M5Stack Tab5 (ESP32-P4) over WiFi","archived":false,"fork":false,"pushed_at":"2026-05-06T02:35:06.000Z","size":86,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-06T04:35:28.442Z","etag":null,"topics":["embedded","esp-idf","esp32","esp32-p4","iot","m5stack","mjpeg","raspberry-pi","riscv","tab5","video-streaming"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmatking.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-20T02:17:15.000Z","updated_at":"2026-04-26T11:13:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dmatking/m5stack-tab5-video-stream","commit_stats":null,"previous_names":["dmatking/m5stack-tab5-video-stream"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dmatking/m5stack-tab5-video-stream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmatking%2Fm5stack-tab5-video-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmatking%2Fm5stack-tab5-video-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmatking%2Fm5stack-tab5-video-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmatking%2Fm5stack-tab5-video-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmatking","download_url":"https://codeload.github.com/dmatking/m5stack-tab5-video-stream/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmatking%2Fm5stack-tab5-video-stream/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33833277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-02T02:00:07.132Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embedded","esp-idf","esp32","esp32-p4","iot","m5stack","mjpeg","raspberry-pi","riscv","tab5","video-streaming"],"created_at":"2026-06-02T19:01:11.453Z","updated_at":"2026-06-02T19:01:12.552Z","avatar_url":"https://github.com/dmatking.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# M5Stack Tab5 Video Stream\n\nMJPEG video + synchronized PCM audio streaming over WiFi to the M5Stack Tab5 (ESP32-P4).\n\nThe server pre-extracts frames and audio from any YouTube video (via yt-dlp) or local\nfile into a disk cache, then serves them over HTTP. The firmware fetches frames and audio\nchunks on demand, decodes JPEG in hardware, rotates via PPA, and plays audio through the\nES8388 codec — all with A/V sync locked to wall clock.\n\n---\n\n## Hardware\n\n| Component | Detail |\n|-----------|--------|\n| Board | M5Stack Tab5 |\n| SoC | ESP32-P4 (dual-core RISC-V 400 MHz) |\n| WiFi | ESP32-C6 co-processor via SDIO |\n| Display | 5\" 1280×720 MIPI-DSI (portrait framebuffer) |\n| Audio | ES8388 codec, onboard speaker |\n\n---\n\n## Server setup\n\nThe server runs on any Linux machine with Python 3, ffmpeg, and yt-dlp. A Raspberry Pi 5\nworks well and is what this was developed on.\n\n### Install dependencies\n\n```bash\npip3 install flask gunicorn yt-dlp\n# ffmpeg via system package manager, e.g.:\nsudo apt install ffmpeg\n```\n\n### Configure channels\n\nEdit `server/channels.json` — each key is a channel name, value is a YouTube URL\nor a path to a local video file:\n\n```json\n{\n  \"my_channel\": \"https://www.youtube.com/watch?v=...\"\n}\n```\n\nOn first request the server resolves the URL with yt-dlp and extracts all frames\nand audio into `server/cache/\u003cchannel\u003e/`. Subsequent runs serve from cache instantly.\n\n### Run the server\n\n```bash\ncd server\ngunicorn -w 2 -b 0.0.0.0:8080 server:app\n```\n\nThe server will begin extraction in the background on the first request. Video playback\nstarts as soon as the first frames are available — you don't need to wait for the full\nvideo to be extracted.\n\n---\n\n## Firmware setup\n\n### Prerequisites\n\n- ESP-IDF 5.5.3 (`~/esp/esp-idf-v5.5.3` or set `IDF_PATH`)\n- WiFi credentials in `~/.esp_creds`:\n\n```\nCONFIG_WIFI_SSID=\"YourNetwork\"\nCONFIG_WIFI_PASS=\"YourPassword\"\n```\n\n### Configure\n\n```bash\nidf.py menuconfig\n# → Video Stream Config\n#   SERVER_IP  — IP address of the machine running the server\n#   SERVER_PORT — 8080 by default\n#   CHANNEL    — must match a key in channels.json\n```\n\n### Build and flash\n\n```bash\nidf.py build\nidf.py flash\n```\n\n---\n\n## Architecture\n\n### HTTP pull model\n\nThe firmware requests data on demand rather than the server pushing a stream. This\ntolerates WiFi hiccups gracefully — a missed frame is simply retried on the next\nrequest.\n\n```\nESP32-P4                          Server (Pi)\n─────────────────────────────     ────────────────────────────\nGET /frame/\u003cchannel\u003e/\u003cms\u003e   ───►  serve frame_NNNNN.jpg from disk\nGET /audio/\u003cchannel\u003e/\u003cs\u003e/\u003cn\u003e───►  serve raw u8 PCM slice from audio.raw\nGET /info                   ───►  channel metadata (duration, fps, etc.)\n```\n\n### Video pipeline (ESP32-P4)\n\n```\n[fetch task, core 1]          [decode task, core 0]\n  HTTP GET /frame               xQueueReceive(ready_q)\n  → JPEG in PSRAM slot          HW JPEG decode → RGB565\n  → xQueueSend(ready_q)         PPA rotate 90° CW → framebuffer\n  ← xQueueReceive(free_q)       board_lcd_commit()  (double-buffer flip)\n                                vTaskDelayUntil(50ms)  ← paces to 20fps\n```\n\n16 pipeline slots provide ~800 ms of buffer to absorb WiFi retransmit spikes.\n\n### A/V sync\n\nBoth audio and video reference wall clock from the moment the first frame is\nsuccessfully fetched. Audio samples are consumed by the I2S DMA at exactly\n16 kHz — any drift in the fetch rate shows up as silence (not desync).\n\n### Display\n\nFrames are extracted at 992×560 (landscape) and rotated 90° CW on-device via the\nPPA hardware accelerator, then letterboxed into the 720×1280 portrait framebuffer.\nDouble buffering (2 hardware DPI framebuffers) eliminates tearing.\n\n### Server pre-processing\n\nffmpeg extracts frames at 20 fps and audio as mono unsigned 8-bit PCM at 16 kHz.\nOn Raspberry Pi 5, H.265 sources use hardware decode (`hevc_v4l2m2m`);\nH.264/VP9 fall back to software (the Pi 5 CPU handles this at these resolutions).\n\n---\n\n## Notes\n\n**The channel to play is hardcoded in the firmware.** It is set via `CHANNEL` in\n`menuconfig` (or `sdkconfig.defaults`) and compiled in. To switch to a different video,\nupdate the channel name, rebuild, and reflash.\n\n---\n\n## TODO\n\n- **Play / pause and volume controls** — use the Tab5's onboard buttons or touchscreen\n  to pause playback and adjust volume without reflashing\n- **On-device channel selection** — browse and switch channels directly from the Tab5\n  touchscreen, no server interaction or reflash required\n- **Server web interface** — a browser UI to add new videos (YouTube URLs or local\n  files), monitor extraction progress, and manage the channel list\n\n---\n\n## Tuning\n\n| Parameter | Location | Effect |\n|-----------|----------|--------|\n| `FPS` | `server/server.py` | Extraction frame rate (default 20) |\n| `PIPELINE_SLOTS` | `main/main.c` | Pre-fetch buffer depth (default 16 = ~800 ms) |\n| `AUDIO_CHUNK_SAMPLES` | `main/main.c` | Audio fetch granularity (default 1600 = 100 ms) |\n| `JPEG_IN_MAX` | `main/main.c` | Max compressed JPEG size per frame (default 128 KB) |\n| `SRC_W` / `SRC_H` | `main/main.c` | Frame dimensions — must be divisible by 8 |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmatking%2Fm5stack-tab5-video-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmatking%2Fm5stack-tab5-video-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmatking%2Fm5stack-tab5-video-stream/lists"}