{"id":51578841,"url":"https://github.com/zacs/esphome-person_detector","last_synced_at":"2026-07-11T04:00:27.275Z","repository":{"id":370028178,"uuid":"1290769496","full_name":"zacs/esphome-person_detector","owner":"zacs","description":null,"archived":false,"fork":false,"pushed_at":"2026-07-08T02:05:24.000Z","size":79,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-08T02:06:58.425Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zacs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-06T08:15:44.000Z","updated_at":"2026-07-08T02:05:29.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/zacs/esphome-person_detector","commit_stats":null,"previous_names":["zacs/esphome-person_detector"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/zacs/esphome-person_detector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zacs%2Fesphome-person_detector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zacs%2Fesphome-person_detector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zacs%2Fesphome-person_detector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zacs%2Fesphome-person_detector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zacs","download_url":"https://codeload.github.com/zacs/esphome-person_detector/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zacs%2Fesphome-person_detector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35350133,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-11T02:00:05.354Z","response_time":104,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-11T04:00:17.478Z","updated_at":"2026-07-11T04:00:27.265Z","avatar_url":"https://github.com/zacs.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# esphome-person_detector\n\nOn-device human-presence detection for ESPHome.\n\n`person_detect` is a general-purpose ESPHome external component that uses a\ncamera to decide whether a person is in frame — presence only, not identity —\nentirely on-device, and exposes it to Home Assistant as an occupancy\n`binary_sensor`.\n\nIt's built to be portable and extended: the detection model, the camera sensor,\nthe frame-source backend, and the target SoC are all swappable (see\n[Extending and portability](#extending-and-portability)). The Seeed Studio\nreTerminal D1001 (ESP32-P4 + MIPI-CSI SC2356) is simply the first board that's\nbeen verified end-to-end on hardware — a starting point, not the purpose. Other\nESP-DL targets, such as an ESP32-S3 with a DVP camera, fit the same design; they\nwork today and just need verifying.\n\n## Privacy\n\nNothing leaves the device. There's no cloud API, no companion server, and no\nstreaming to Home Assistant — all inference runs locally. It uses no\nface-recognition models and stores no biometric data; it answers one question,\n\"is a person in frame right now?\", and nothing else. An optional `switch` turns\ndetection on and off: on is the default, and turning it off releases the camera\nand stops all inference, so flipping it off gives you a hard privacy cut.\n\n## The model\n\nDetection uses Espressif's\n[`pedestrian_detect`](https://components.espressif.com/components/espressif/pedestrian_detect)\nmodel (v0.3.0), built on [ESP-DL](https://github.com/espressif/esp-dl),\nEspressif's on-device deep-learning library. It's an INT8-quantized full-body\npedestrian detector with a 224×224×3 input, pulled in automatically by the\nESP-IDF component manager — you don't fetch or embed anything by hand.\n\nTwo practical consequences worth knowing:\n\n- It looks for whole standing or walking people, not faces. A close-up of your\n  face may not trigger it, but a person standing in the room will — so aim the\n  camera to see bodies, not head-and-shoulders.\n- Inference takes about 75 ms on the ESP32-P4 (it also runs on the S3, slower).\n\nESP-DL ships a whole [catalog of models](https://github.com/espressif/esp-dl/tree/master/models),\nand the right one depends on framing. `pedestrian_detect` (the default) suits\nroom-scale presence, where you see whole bodies. For close range — a desk, a\ndoorway, a doorbell, where the camera mostly sees a head and shoulders —\n[`human_face_detect`](https://github.com/espressif/esp-dl/tree/master/models/human_face_detect)\ntends to work better. It detects that a face is *present*, not whose it is\n(that's face *detection*, not *recognition*), so it stays within the\npresence-only, no-identity design. Others like the 80-class `coco_detect` also\ninclude a person class.\n\nToday only `pedestrian` is wired up as a `model:` value; adding another ESP-DL\ndetector (face, COCO, hand, …) is the small extension described in\n[Extending and portability](#extending-and-portability) — any model that returns\n`dl::detect::result_t` boxes drops into the existing pipeline.\n\n## Requirements\n\nThe detector runs anywhere ESP-DL does; these are the practical constraints:\n\n| | |\n|---|---|\n| SoC | ESP32-P4 is verified. Other ESP-DL targets (e.g. ESP32-S3) are allowed but experimental — a warning, not a build error. See [Extending and portability](#extending-and-portability) |\n| Framework | ESP-IDF (no Arduino core on these targets) |\n| PSRAM | Required — the model runtime and frame buffers live here |\n| Flash | `flash_size: 16MB` — the embedded model makes the app image ~2.3 MB; 16 MB lets ESPHome auto-size an app partition that fits, with no custom partition CSV |\n\nTested against ESPHome 2026.6.0 / ESP-IDF v5.5.4.\n\n## Quick start\n\nThese paths use the reTerminal D1001 because it's the verified board, but the\nshape is the same for any board: pull the component, give it a frame source, and\nexpose a sensor. For a different ESP32-P4 camera board, point `esp_video_camera`\nat your sensor and pins; for another SoC, see [Extending and portability](#extending-and-portability).\n\n### Option A — flash the prebuilt D1001 image (fastest)\n\nGrab `reterminal-presence.factory.bin` from the [latest\nrelease](https://github.com/zacs/esphome-person_detector/releases), flash it to\noffset `0x0` (the [ESPHome web flasher](https://web.esphome.io) or\n`esptool.py --chip esp32p4 write_flash 0x0 …`), then join the \"reTerminal\nPresence\" Wi-Fi AP / captive portal (or Improv over USB) to set your network.\n\n### Option B — start from the example config\n\n[`example/reterminal_d1001.yaml`](example/reterminal_d1001.yaml) is a complete,\ncompilable D1001 config — board bring-up, camera wiring, and detector. Copy it,\ndrop in your Wi-Fi, and build:\n\n```bash\npip install \"esphome==2026.6.0\"\nesphome run example/reterminal_d1001.yaml\n```\n\n### Option C — add it to an existing ESP32-P4 config\n\nPull the component in, point the detector at a camera backend, and expose a\nsensor. On the D1001 the camera backend is `esp_video_camera` (CSI + ISP →\nRGB888); its power and reset lines sit on an XL9535 I²C expander, so copy the\n`i2c:`, `xl9535:`, and `esp_video_camera:` blocks from the example for the full\nwiring. The detector part is just:\n\n```yaml\nexternal_components:\n  - source: github://zacs/esphome-person_detector@v0.2.0   # pin a released tag\n    components: [person_detect, esp_video_camera]\n\nperson_detect:\n  id: presence\n  frame_source_id: d1001_cam   # id of your esp_video_camera (see below)\n\nbinary_sensor:\n  - platform: person_detect\n    person_detect_id: presence\n    name: \"Room Occupied\"       # device_class defaults to \"occupancy\"\n```\n\nA quick note on `frame_source_id` vs `camera_id`: use `frame_source_id` to point\nat an `esp_video_camera` (the raw MIPI-CSI path, which is what the D1001 uses),\nand `camera_id` only if you're feeding a stock ESPHome JPEG `camera:` instead.\nSet exactly one — see [Frame sources](#frame-sources).\n\n## Configuration\n\n### `esp_video_camera:` — the ESP32-P4 MIPI-CSI backend\n\nOwns the sensor and hands the detector upright RGB888 frames.\n\n```yaml\nesp_video_camera:\n  id: d1001_cam\n  sda: 37                # sensor SCCB (I²C) pins — D1001: 37/38\n  scl: 38\n  i2c_port: 1            # SCCB I²C controller; must differ from your `i2c:` bus\n  # i2c_id: touch_bus    # OR: share an existing i2c bus instead of sda/scl/i2c_port\n  resolution: 1280x720\n  rotation: auto         # IMU-picked at boot; or pin 0 / 90 / 180 / 270\n  imu:                   # only needed for rotation: auto\n    i2c_id: bus_expander\n    address: 0x6A        # D1001 LSM6DS3TR\n  # exposure: 811        # raw sensor units; omit for a bright auto default\n  # gain: 63             # raw sensor units; omit for auto\n  # power/reset lines (D1001 wires these to the XL9535 expander):\n  enable_pin: { xl9535: cam_expander, number: 1 }\n  powerdown_pin: { xl9535: cam_expander, number: 3 }\n  reset_pin: { xl9535: cam_expander, number: 11 }\n```\n\n- `rotation` accepts `auto` or an explicit `0` / `90` / `180` / `270`. Rotation\n  is counter-clockwise, so `90` turns the image 90° CCW (confirmed on the D1001).\n  `auto` reads the accelerometer once at boot and picks the orientation that\n  keeps a standing person upright, which is handy for a device you reposition; an\n  explicit value always wins and is best for a fixed mount. Auto needs the device\n  roughly upright — lying flat, gravity is straight down and it holds the last\n  orientation.\n- `exposure` and `gain` default to a bright value because the SC2356 powers up\n  near-black. If a room comes out too dark or too bright, set them (raw sensor\n  units; the valid range is logged at boot).\n- SCCB bus — by default the camera installs its own I²C master on `sda`/`scl` /\n  `i2c_port` (`i2c_port` must differ from every ESPHome `i2c:` bus). On a board\n  that already uses both P4 I²C controllers — e.g. a display + touch build where\n  the SCCB physically shares the touch bus wires — set `i2c_id:` to an existing\n  `i2c:` bus instead: the camera then joins that bus as another device rather\n  than installing a second master. `i2c_id` and `sda`/`scl`/`i2c_port` are\n  mutually exclusive; the sensor keeps its own `frequency` (per-device on the\n  shared bus, so it's independent of e.g. a 400 kHz touch controller on the same\n  wires).\n\n### `person_detect:` — the detector\n\n```yaml\nperson_detect:\n  id: presence\n  frame_source_id: d1001_cam  # an esp_video_camera; OR camera_id: for a JPEG camera\n  interval: 1500ms            # inference cadence (default 1500ms, min 100ms)\n  confidence_threshold: 60%   # min score to count a detection (default 60%)\n  clear_after: 3              # consecutive misses before clearing (default 3)\n  model: pedestrian           # selectable model (default: pedestrian)\n  # advanced task tuning (defaults shown):\n  task_priority: 2            # keep below a UI/LVGL task\n  task_stack_size: 8192\n  task_core: 1\n  on_person_detected:\n    - logger.log: \"Person detected\"\n  on_person_cleared:\n    - logger.log: \"Room empty\"\n```\n\n### `binary_sensor` / `sensor` / `switch`\n\n```yaml\nbinary_sensor:\n  - platform: person_detect\n    person_detect_id: presence\n    name: \"Room Occupied\"\n    filters:\n      - delayed_off: 15s        # optional hold-open on top of clear_after\n\nsensor:                          # optional\n  - platform: person_detect\n    person_detect_id: presence\n    type: confidence             # best-box score, 0–100 %\n    name: \"Presence Confidence\"\n  - platform: person_detect\n    person_detect_id: presence\n    type: count                  # boxes over threshold\n    name: \"People Count\"\n\nswitch:                          # optional detection on/off (privacy)\n  - platform: person_detect\n    person_detect_id: presence\n    name: \"Presence Detection\"   # on = detecting (default); off = camera idle, no inference\n```\n\n## Frame sources\n\n`person_detect` takes frames from exactly one backend:\n\n| Key | Backend | Frames | Use when |\n|---|---|---|---|\n| `frame_source_id` | `esp_video_camera` (this repo) | raw RGB888 (CSI + ISP + PPA) | ESP32-P4 MIPI-CSI sensors (SC2356 / D1001) — the path the D1001 uses |\n| `camera_id` | a stock ESPHome `camera:` | JPEG, decoded on-device | a board that already exposes a JPEG camera to ESPHome |\n\nBoth sit behind one `FrameSource` seam ([`DESIGN.md`](DESIGN.md) §2). The\n`camera_id` path exists for composability, but ESPHome (as of 2026.6.0) ships no\nuser-configurable MIPI-CSI camera platform, so on the P4 you use\n`esp_video_camera`.\n\n## Coexisting with an LVGL / display config\n\nInference runs on its own low-priority, pinned FreeRTOS task (`task_priority`,\n`task_core`), publishes only from the main loop, and keeps every large buffer in\nPSRAM, so it stays out of the way of an 800×1280 MIPI-DSI LVGL UI. If your UI\ntask is pinned to a core, pin detection to the other with `task_core`.\n\n## Tuning\n\n- `confidence_threshold` — raise it (70–80 %) to cut false positives from\n  clutter, or lower it (45–55 %) if real people are missed.\n- `interval` — presence doesn't need video framerates; 1–2 s is a good default.\n  Inference is ~75 ms on the P4, so the task idles most of the interval.\n- `clear_after` + `delayed_off` — `clear_after` debounces single-frame misses; a\n  `delayed_off:` filter adds occupancy-style hold-open.\n- Exposure and lighting — the SC2356 is a small 2 MP sensor. If confidence sags\n  in dim rooms, raise `exposure`/`gain`, and avoid strong backlight (a window\n  behind the subject) that silhouettes people.\n\n## Diagnostics\n\nWith `logger: level: DEBUG`, each inference logs its time, frame dimensions, box\ncount, best score, a min/max/mean frame-brightness probe, and free PSRAM.\n`dump_config` prints the model, cadence, threshold, task placement, PSRAM\ncost/low-water, last inference time, and capture-failure count. On boot,\n`esp_video_camera` logs the SCCB port, the applied exposure/gain with their\nranges, and — for `rotation: auto` — the raw accelerometer reading and the\nrotation it chose.\n\n## Memory / flash budget\n\nAll large buffers live in PSRAM; internal SRAM use is limited to the inference\ntask stack. Measured static footprint (CI `esphome compile`, ESP32-P4, ESPHome\n2026.6.0 / ESP-IDF 5.5.4, `esp_video_camera` + `person_detect` + ESP-DL +\npedestrian model):\n\n| Metric | Value |\n|---|---|\n| Total image | ~2.29 MB (Flash 13.6 % of 16 MB) |\n| `.rodata` (model weights + const) | 596 KB |\n| Internal RAM (static) | ~5.8 % of 512 KB |\n| PPA / capture / tensor-arena buffers | PSRAM, allocated at runtime (not in the image) |\n\nRuntime PSRAM depends on resolution and rotation; read it on your build from the\n`model runtime PSRAM cost` (startup) and `PSRAM free low-water` (`dump_config`)\nlog lines. Roughly: model weights go in flash; the ESP-DL tensor arena, the\nRGB888 frame (1280×720 ≈ 2.6 MB), and camera buffers go in PSRAM; the task stack\nis internal SRAM.\n\n## How it works\n\n`PersonDetector` pulls one typed RGB frame per `interval` from a `FrameSource`\n(on a dedicated low-priority FreeRTOS task), runs the ESP-DL model on it (resized\nto the model input via the P4's hardware image path), and marshals the result to\nthe main loop for debouncing, state publishing, and triggers. See\n[`DESIGN.md`](DESIGN.md) for the full design and [`BRINGUP.md`](BRINGUP.md) for a\nfirst-flash checklist.\n\n## Extending and portability\n\nThe component is layered so you can add support along several axes without\ntouching the core — including the target SoC, which is really just another axis:\n\n- Detection model — pick from [ESP-DL's model catalog](https://github.com/espressif/esp-dl/tree/master/models)\n  (pedestrian, face, hand, the 80-class COCO detector, and more). Add one with a\n  row in the `MODELS` table plus a `case` in `PersonDetector::create_model_()`\n  (`components/person_detect/`); any ESP-DL detector returning\n  `dl::detect::result_t` boxes fits.\n- Camera sensor — a `sensor:` enum entry and its `esp_cam_sensor` Kconfig in\n  `components/esp_video_camera/__init__.py`; the capture/PPA path is\n  sensor-independent.\n- Frame-source backend — implement `person_detect::FrameSource`\n  (`frame_source.h`) and yield `FrameView`s; the detector consumes them unchanged.\n- Board — a device YAML with its `esp32:` details, camera wiring, and a\n  `person_detect` block. The D1001 files under `example/` and `firmware/` are the\n  first; others sit alongside them.\n- Target SoC — the detector is SoC-agnostic (ESP-DL runs on the S3 as well as\n  the P4), so `person_detect` only warns off-P4 rather than blocking. The\n  P4-specific piece is `esp_video_camera`, which uses the P4's MIPI-CSI + ISP +\n  PPA and hard-fails elsewhere. To run on an S3, add a frame-source backend fed\n  by a DVP JPEG camera — ESPHome's `esp32_camera` (e.g. an OV2640) via\n  `camera_id`. Only the P4 path is hardware-verified; the S3 route exists behind\n  the same seam but needs bring-up (slower inference, no hardware rotation).\n\nContributions for new models, sensors, boards, and the S3 path are welcome.\n\n## License\n\nC++ is licensed GPL-3.0-only to match the ESPHome ecosystem (see `LICENSE`).\nThe bundled Espressif ESP-DL and pedestrian model are Apache-2.0 / MIT — see\n`NOTICE`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzacs%2Fesphome-person_detector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzacs%2Fesphome-person_detector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzacs%2Fesphome-person_detector/lists"}