{"id":20161649,"url":"https://github.com/openipc/pixelpilot_rk","last_synced_at":"2025-10-09T10:13:52.491Z","repository":{"id":255435596,"uuid":"849552314","full_name":"OpenIPC/PixelPilot_rk","owner":"OpenIPC","description":"Application that decodes an RTP Video Stream and displays it on screen, for Rockchip devices.","archived":false,"fork":false,"pushed_at":"2025-10-06T09:02:20.000Z","size":1951,"stargazers_count":40,"open_issues_count":14,"forks_count":27,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-10-06T11:14:55.604Z","etag":null,"topics":["fpv","openipc","openipc-fpv","pixelpilot","rockchip","urllc"],"latest_commit_sha":null,"homepage":"https://openipc.org","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenIPC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"open_collective":"openipc"}},"created_at":"2024-08-29T19:49:53.000Z","updated_at":"2025-10-06T09:02:27.000Z","dependencies_parsed_at":"2024-10-27T23:09:08.662Z","dependency_job_id":"33059703-557c-4f82-9726-d715a40b7b6d","html_url":"https://github.com/OpenIPC/PixelPilot_rk","commit_stats":{"total_commits":98,"total_committers":9,"mean_commits":10.88888888888889,"dds":0.5714285714285714,"last_synced_commit":"94e36063f8ad3a83d0b0a95e25d8454d029d2c2b"},"previous_names":["openipc/pixelpilot_rk"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/OpenIPC/PixelPilot_rk","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenIPC%2FPixelPilot_rk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenIPC%2FPixelPilot_rk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenIPC%2FPixelPilot_rk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenIPC%2FPixelPilot_rk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenIPC","download_url":"https://codeload.github.com/OpenIPC/PixelPilot_rk/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenIPC%2FPixelPilot_rk/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001323,"owners_count":26083040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fpv","openipc","openipc-fpv","pixelpilot","rockchip","urllc"],"created_at":"2024-11-14T00:19:57.387Z","updated_at":"2025-10-09T10:13:52.484Z","avatar_url":"https://github.com/OpenIPC.png","language":"C","funding_links":["https://opencollective.com/openipc"],"categories":[],"sub_categories":[],"readme":"# PixelPilot_rk\n\u003e [!IMPORTANT]\n\u003e Warning, this is an experimental project.\n\u003e\n\u003e Use this software at your own risk.\n\n## Introduction\n\nWFB-ng client (Video Decoder) for Rockchip platform powered by the [Rockchip MPP library](https://github.com/rockchip-linux/mpp).\nIt also displays a simple cairo based OSD that shows the bandwidth, decoding latency, and framerate of the decoded video, and wfb-ng link statistics.\n\nThis project is based on a unique frozen development [FPVue_rk](https://github.com/gehee/FPVue_rk) by [Gee He](https://github.com/gehee).\n\nTested on RK3566 (Radxa Zero 3W) and RK3588s (Orange Pi 5).\n\n## Compilation\n\nBuild on the Rockchip linux system directly.\n\n## Install dependencies\n\n- drm, cairo, mpp, logging, json, msgpack, gpiod, yaml-cpp\n\n```\nsudo apt install libdrm-dev libcairo-dev librockchip-mpp-dev libspdlog-dev nlohmann-json3-dev libmsgpack-dev libgpiod-dev libyaml-cpp-dev\n```\n\n- gstreamer\n\n```\nsudo apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev\n```\n\n## Build Instructions\n\nBuild and run application in production environment:\n\n```\ngit clone https://github.com/OpenIPC/PixelPilot_rk\ncd PixelPilot_rk\ngit submodule update --init\ncd ..\n```\n\n```\ncmake -B build\nsudo cmake --build build --target install\nbuild/pixelpilot\n```\n\nBuild and run application for debugging purposes:\n\n```\ncmake -B build -DCMAKE_BUILD_TYPE=Debug\ncmake --build build\nbuild/pixelpilot --osd\n```\n\nBuild and run gsmenu SDL simulator locally\n```\n./sim.sh\n```\nSimulator has w,a,s,d,Enter input.\nPress t to toggle drone detection.\n\n## Usage\n\nShow command line options:\n```\npixelpilot --help\n```\n\n### OSD config\n\nOSD is set-up declaratively in `/etc/pixelpilot/config_osd.json` file (or whatever is set via `--osd-config`\ncommand line key.\n\nOSD is described as an array of widgets which may subscribe to fact updates (they receive each fact\nupdate they subscribe to) and those widgets are periodically rendered on the screen (in the order they\ndeclared in config). So the goal is that widgets would decide how they should be rendered based on\nthe values of the facts they are subscribed to. If widget needs to render not the latest value of the\nfact, but some processed value (like average / max / total etc), the widget should keep the necessary\nstate for that. There is a helper class `MovingAverage` that would be helpful to calculate common\nstatistical parameters.\n\nEach fact has a specific datatype: one of `int` (signed integer) / `uint` (unsigned integer) /\n`double` (floating point) / `bool` (true/false) / `string` (text). Type cast is currently not\nimplemented, so it is important to use the right type in the widget code and templates.\n\nFacts may also have tags: a set of string key-\u003evalue pairs. Widget may filter facts by tags as well as by name.\nCurrently there are several generic OSD widgets and several specific ad-hoc ones. There are quite a\nlot of facts to which widgets can subscribe to:\n\n| Fact                           | Type | Description                                                               |\n|:-------------------------------|:-----|:--------------------------------------------------------------------------|\n| `dvr.recording`                | bool | Is DVR currently recording?                                               |\n| `video.width`                  | uint | The width of the video stream                                             |\n| `video.height`                 | uint | The height of the video stream                                            |\n| `video.displayed_frame`        | uint | Published  with value \"1\" each time a new video frame is displayed        |\n| `video.decode_and_handover_ms` | uint | Time from the moment packet is received to time it is displayed on screen |\n| `video.decoder_feed_time_ms`   | uint | Time to feed the video packet to hardware decoder                         |\n| `gstreamer.received_bytes`     | uint | Number of bytes received from gstreamer (published for each packet)       |\n| `osd.custom_message`           | str  | The custom message passed via `--osd-custom-message` feature              |\n\nThere are many facts based on Mavlink telemetry, see `mavlink.c`. All of them have tags \"sysid\" and\n\"compid\", but some have extra tags.\nCurrently implemented fact categories are grouped by Mavlink message types:\n\n| Fact                                | Type     | Description                                                                                                                                                                                                                       |\n|:------------------------------------|:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `mavlink.heartbeet.base_mode.armed` | bool     | Is drone armed?                                                                                                                                                                                                                   |\n| `mavlink.raw_imu.*`                 | int      | Raw values from gyroscope and accelerometer, See [RAW_IMU](https://mavlink.io/en/messages/common.html#RAW_IMU)                                                                                                                    |\n| `mavlink.sys_status.*`              | int/uint | Some of the fields from [SYS_STATUS](https://mavlink.io/en/messages/common.html#SYS_STATUS)                                                                                                                                       |\n| `mavlink.battery_status.*`          | int      | Some of the fields from [BATTERY_STATUS](https://mavlink.io/en/messages/common.html#BATTERY_STATUS)                                                                                                                               |\n| `mavlink.rc_channels_raw.chanN`     | uint     | Raw values of remote control chanels (N is from 1 to 8)                                                                                                                                                                           |\n| `mavlink.gps_raw.*`                 | int/uint | Raw data from GNSS sensor, see [GPS_RAW_INT](https://mavlink.io/en/messages/common.html#GPS_RAW_INT)                                                                                                                              |\n| `mavlink.vfr_hud.*`                 | double   | Metrics common for fixed wing OSDs, see [VFR_HUD](https://mavlink.io/en/messages/common.html#VFR_HUD)                                                                                                                             |\n| `mavlink.global_position_int.*`     | int      | Position estimation based on sensor fusion, see [GLOBAL_POSITION_INT](https://mavlink.io/en/messages/common.html#GLOBAL_POSITION_INT)                                                                                             |\n| `mavlink.attitude.*`                | double   | See [ATTITUDE](https://mavlink.io/en/messages/common.html#ATTITUDE)                                                                                                                                                               |\n| `mavlink.radio_status.*`            | uint/int | Status of various radio equipment. Tags `{sysid: 3, compid: 68}` encode the [injected status of WFB-ng receiver](https://github.com/svpcom/wfb-ng/blob/4ea700606c259960ea169bad1f55fde77850013d/wfb_ng/conf/master.cfg#L227-L228) |\n\nMore can be easily added later. You can use `DebugWidget` to inspect the current raw value of the fact(s).\n\nPixelpilot is also able to connect to WFB-ng statistics API and extract some of the facts from there.\nReceiving packets statistics (each fact has \"id\" tag - channel name, eg \"video\"/\"mavlink\"/\"tunnel\" etc),\nthe last section of the name is either `delta` or `total`:\n\n| Fact                                | Type | Description                                  |\n|:------------------------------------|:-----|:---------------------------------------------|\n| `wfbcli.rx.packets.all.delta|total` | uint | Number of packets received                   |\n| `wfbcli.rx.packets.all_bytes.*`     | uint | Number of bytes received                     |\n| `wfbcli.rx.packets.dec_err.*`       | uint | Number of packets we failed to decrypt       |\n| `wfbcli.rx.packets.session.*`       | uint | Number of session packets received           |\n| `wfbcli.rx.packets.data.*`          | uint | Number of payload packets received           |\n| `wfbcli.rx.packets.uniq.*`          | uint | Number of unique packets received            |\n| `wfbcli.rx.packets.fec_rec.*`       | uint | Number of packets recovered with FEC         |\n| `wfbcli.rx.packets.lost.*`          | uint | Number of packets we failed recover with FEC |\n| `wfbcli.rx.packets.bad.*`           | uint | Number of packets lost due to internal errors|\n| `wfbcli.rx.packets.out.*`           | uint | Number of good packets                       |\n| `wfbcli.rx.packets.out_bytes.*`     | uint | Number of good bytes                         |\n\nReceiving per-antenna statistics (each fact has \"id\" - channel name and \"ant_id\" - antenna number tags)\n| Fact                           | Type   | Description                                     |\n|:-------------------------------|:-------|:------------------------------------------------|\n| `wfbcli.rx.ant_stats.freq`     | uint   | Antenna frequency, MHz, eg 5800                 |\n| `wfbcli.rx.ant_stats.mcs`      | uint   | MCS value for this antenna                      |\n| `wfbcli.rx.ant_stats.bw`       | uint   | Bandwidth value for antenna (MHz)               |\n| `wfbcli.rx.ant_stats.pkt_recv` | uint   | Number of WiFi packets received by this antenna |\n| `wfbcli.rx.ant_stats.rssi_avg` | int    | Average RSSI for this antenna                   |\n| `wfbcli.rx.ant_stats.snr_avg`  | double | Average SNR for this antenna                    |\n\nTransmitting packets stats (same tags as receiving packets), add `delta` or `total` in the end:\n\n| Fact                                     | Type | Description                              |\n|:-----------------------------------------|:-----|:-----------------------------------------|\n| `wfbcli.tx.packets.injected.delta|total` | uint | Number of successfully injected packets  |\n| `wfbcli.tx.packets.injected_bytes.*`     | uint | Number of successfully injected bytes    |\n| `wfbcli.tx.packets.dropped.*`            | uint | Number of dropped packets                |\n| `wfbcli.tx.packets.truncated.*`          | uint | Number of truncated (?) packets          |\n| `wfbcli.tx.packets.fec_timeouts.*`       | uint | ?                                        |\n| `wfbcli.tx.packets.incoming.*`           | uint | Even TX interface may receive, n packets |\n| `wfbcli.tx.packets.incoming_bytes.*`     | uint | Even TX interface may receive, n bytes   |\n\nRF chip temperature, tagged with `ant_id`:\n\n| Fact                     | Type | Description                        |\n|:-------------------------|:-----|:-----------------------------------|\n| `wfbcli.rf_temperature`  | uint | Temperature of RX chip per-antenna |\n\n\n#### Widgets\n\nCurrently we have generic widgets and more ad-hoc specific ones. Generic widgets normally can be used\nto display any fact (as long as datatype matches):\n\n* `TextWidget` - displays a static string of text\n* `IconTextWidget` - displays a graphical icon followed by a static text\n* `TplTextWidget` - displays a string of text by replacing placeholders with the fact values\n* `IconTplTextWidget` - displays a graphical icon followed by templatized text string\n* `BoxWidget` - displays a static square. Might be good as a background.\n* `BarChartWidget` - displays a simple bar chart for the single fact's statistics. Each bar represents\n either minimum or maximum or sum or count or average of the fact over time interval. Can be used to show\n eg the average video bitrate or RSSI or FPS.\n* `PopupWidget` - displays a stacked pop-ups with text facts which fade-away after timeout.\n* `DebugWidget` - displays debug information (name, type, tags, value) about fact(s)\n* `IconSelectorWidget` - display a icon based on a fact's value\n\nSpecific widgets expect quite concrete facts as input:\n\n* `DvrStatusWidget` - shows up when DVR is recording and is hidden when not.\n  Uses `dvr.recording` fact\n* `VideoWidget` - shows FPS and video resolution.\n  Uses `video.displayed_frame`, `video.width`, `video.height` facts\n* `VideoBitrateWidget` - shows video bitrate (not radio link, but video!).\n  Uses `gstreamer.received_bytes` fact\n* `VideoDecodeLatencyWidget` - shows video frame decode and display latency (avg/min/max).\n  Uses `video.decode_and_handover_ms` fact\n* `GPSWidget` - displays GPS fix type (no fix / 2D fix / 3D fix etc) and GPS coordinates.\n  Uses `mavlink.gps_raw.fix_type`, `mavlink.gps_raw.lat` and `mavlink.gps_raw.lon` facts\n\n## GSMenu\n\nThe gsmenu provides a ui to modify air and ground settings.\nNavigation is controlled via a GPIO buttons.\nGPIO mapping can be configured in pixelpilot.yaml, see example.\nPixelPilot_rk will take ownership of the needed gpios.\nSettings on air and ground are get/set useing the gsmenu.sh script.\n\nInstall gsmenu.sh dependencies:\n```\ngit clone https://github.com/OpenIPC/yaml-cli.git\ncd yaml-cli\ncmake -B build\nsudo cmake --build build --target install\ncurl -L -o /usr/local/bin/yq https://github.com/mikefarah/yq/releases/download/v4.45.4/yq_linux_arm64\nchmod +x /usr/local/bin/yq\nsudo apt install drm-info jq\n```\n\n### Navigation\nNavigation mode:\n  - Up/Down: Navigate\n  - Right: Select page / Start edit mode\n  - Left: Back/Close\n\nEdit mode:\n  - Up/down: Change value\n  - Right: Confirm selection\n  - Left: Back/Cancel\n\nKeyboard mode:\n\n  - Up/Down/Left/Right: Select key\n  - Enter: Press selected key\n\nWhen not in menu Up/Down will open the menu.\n\n## Known issues\n\n1. Video is cropped when the fpv feed resolution is bigger than the screen mode.\n1. Crashes when video feed resolution is higher than the screen resolution.\n\n## The way it works\n\nIt uses `gstreamer` to read the RTP media stream from video UDP.\nIt uses `mpp` library to decode MPEG frames using Rockchip hardware decoder.\nIt uses [Direct Rendering Manager (DRM)](https://en.wikipedia.org/wiki/Direct_Rendering_Manager) to\ndisplay video on the screen, see `drm.c`.\nIt uses `mavlink` decoder to read Mavlink telemetry from telemetry UDP (if enabled), see `mavlink.c`\nIt uses `cairo` library to draw OSD elements (if enabled), see `osd.c`.\nIt uses `lvgl`to draw the gsmenu.\nIt writes non-decoded MPEG stream to file as DVR (if enabled) using `minimp4.h` library.\n\nPixelpilot starts several threads:\n\n* main thread:\n  controls gstreamer which reads RTP, extracts MPEG frames and\n  - feeds them to MPP hardware decoder\n  - sends them to DVR thread via mutex-protected `std::queue` (if enabled)\n* DVR_THREAD (if enabled):\n  reads video frames and start/stop/shutdown commands from main thread via `std::queue` and writes\n  frames them to disk using `minimp4` library.\n  It yields on a condition variable for DVR queue\n* FRAME_THREAD:\n  reads decoded video frames from MPP hardware decoder and forwards them to `DISPLAY_THREAD`\n  through DRM `output_list` protected by `video_mutex`.\n  Seems that thread vields on `mpi-\u003edecode_get_frame()` call waiting for HW decoder to return a new frame\n* DISPLAY_THREAD:\n  reads decoded frames and OSD from `video_mutex`-protected `output_list` and calls `drm*` functions to\n  render them on the screen.\n  The loop yields on `video_mutex` and `video_cond` waiting for a new frame to\n  display from FRAME_THREAD\n* MAVLINK_THREAD (if OSD and mavlink configured):\n  reads mavlink packets from UDP, decodes and updates `osd_vars` (without any mutex).\n  The loop yields on UDP read.\n* WFBCLI_THREAD (if OSD is enabled):\n  connects to the local WFB instance stats API, reads JSON stats messages and publishes OSD facts.\n  The loop yields on TCP read.\n* OSD_THREAD (if OSD is enabled):\n  takes `drm_fd`, `output_list` and JSON config as thread parameters,\n  receives Facts through mutex-with-timeout-protected `std::queue`, feeds Facts to widgets and\n  periodically draws widgets on a buffer inside `output_list` using Cairo library.\n  There exists legacy OSD, is based on `osd_vars`, draws using Cairo library, to be removed.\n  The loop yields on queue's mutex with timeout (timeout in order to re-draw OSD at fixed intervals).\n\n## Release\n\n* update project version in `CMakeList.txt`, `project(pixelpilot, VERSION \u003cX.Y.Z\u003e)`, commit\n* push that commit to master (either directly or with PR)\n* tag the tip of the master branch with the same `\u003cX.Y.Z\u003e` version\n* run `git push --tags`; it will publish a new GitHub release\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenipc%2Fpixelpilot_rk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenipc%2Fpixelpilot_rk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenipc%2Fpixelpilot_rk/lists"}