{"id":51040286,"url":"https://github.com/zevisvei/kindle-reading-dashboard","last_synced_at":"2026-06-22T10:03:20.613Z","repository":{"id":365829556,"uuid":"1270937058","full_name":"zevisvei/kindle-reading-dashboard","owner":"zevisvei","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-19T03:54:19.000Z","size":53,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-19T05:26:09.755Z","etag":null,"topics":["kindle"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zevisvei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-16T07:22:15.000Z","updated_at":"2026-06-19T03:54:23.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/zevisvei/kindle-reading-dashboard","commit_stats":null,"previous_names":["zevisvei/kindle-reading-dashboard"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/zevisvei/kindle-reading-dashboard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zevisvei%2Fkindle-reading-dashboard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zevisvei%2Fkindle-reading-dashboard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zevisvei%2Fkindle-reading-dashboard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zevisvei%2Fkindle-reading-dashboard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zevisvei","download_url":"https://codeload.github.com/zevisvei/kindle-reading-dashboard/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zevisvei%2Fkindle-reading-dashboard/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34643624,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kindle"],"created_at":"2026-06-22T10:03:19.747Z","updated_at":"2026-06-22T10:03:20.604Z","avatar_url":"https://github.com/zevisvei.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kindle-reading-dashboard\n\nA local, offline toolkit to **read, decode, and visualize the reading metadata a Kindle stores on-device** — reading time, pace (WPM), per-book progress, highlights, page mapping, and read/unread status — with a clean local web dashboard.\n\nNo cloud account, no Amazon API, no KOReader. Everything is parsed straight from the device's own files over SSH (USB networking) or from a USB mass-storage mount.\n\n\u003e Tested on a **Kindle Basic 10th gen (J9G29R), firmware 5.18.1**, jailbroken with USB networking enabled.\n\n\u003e Discussion / support: [MobileRead thread #373997](https://www.mobileread.com/forums/showthread.php?t=373997).\n\n![dashboard list](docs/screenshot-list.png)\n\n---\n\n## What it does\n\n- **Library list** — every book with title, author, series, % complete, reading time, pace (WPM), current page, and read/reading/unread status. Sortable, searchable, filterable.\n- **Per-book detail** — full metadata: reading-time model, reading-speed distribution, reading timeline, device sessions, highlights/notes, font/display prefs, all `cc.db` fields, and the raw decoded JSON of every source.\n- **Position → page translation** — converts Kindle internal positions to printed page numbers via the `apnx` page map.\n- **Read/unread status** — surfaces the modern \"Mark as Read\" status, which the new Kindle app stores **outside** the legacy library DB (see [docs/read-state-storage.md](docs/read-state-storage.md)).\n\n---\n\n## Where the data comes from\n\nThe Kindle keeps several independent on-device stores. This toolkit reads all of them and joins them per book:\n\n| Data | Source file (on device) | Format | Notes |\n|------|--------------------------|--------|-------|\n| Title, author, series, % progress, legacy read-state | `/var/local/cc.db` | SQLite | The \"Content Catalog\" library DB. See [docs/cc-db-schema.md](docs/cc-db-schema.md). |\n| **Reading time + pace (WPM)** | `\u003cbook\u003e.sdr/*.azw3f` → `timer.model` | KRDS (binary) | The native reader records reading time locally. See [docs/KRDS-format.md](docs/KRDS-format.md). |\n| Total words in book | `*.azw3f` → `book.info.store` | KRDS | |\n| Page-turn history / timeline | `*.azw3f` → `page.history.store` | KRDS | Sparse on FW 5.18.x (often one record/book). |\n| First/last read position | `*.azw3f`/`*.azw3r` → `lpr`/`fpr` | KRDS | |\n| Highlights / notes / bookmarks | `\u003cbook\u003e.sdr/*.azw3r` → `annotation.cache.object` | KRDS | |\n| Font / margins / display prefs | `*.azw3r` → `font.prefs` | KRDS | |\n| Printed page mapping | `*.azw3r` → `apnx.key.oPNToPosition` | KRDS | Index = printed page → starting position. |\n| **Modern \"Mark as Read\"** + device reading sessions | `/mnt/us/system/fmcache/fmcache.db` | SQLite (fast-metrics) | The new app keeps read-state here + the cloud, NOT in `cc.db`. See [docs/read-state-storage.md](docs/read-state-storage.md). |\n\nThe `.sdr` \"sidecar\" folders live next to each book under `/mnt/us/documents/\u003cauthor\u003e/\u003ctitle\u003e.sdr/`.\n\n---\n\n## Install\n\nRequirements:\n- **Python 3** (standard library only, except `paramiko` for SSH).\n- `pip install paramiko`\n\n```bash\ngit clone \u003cthis-repo\u003e\ncd kindle-reading-dashboard\npip install paramiko\n```\n\n### Do you need a jailbroken Kindle?\n\nThe **full dashboard** needs a jailbroken device (SSH/root). Basic sidecar decoding works on a **stock** device mounted as a USB drive:\n\n| Capability | Jailbreak needed? | Why |\n|------------|:---:|-----|\n| Reading time/pace, highlights, page mapping (from `.azw3f`/`.azw3r` sidecars) | **No** | The `.sdr` folders live in `documents/`, readable on a plain USB mount. Use the CLI scripts or `--local D:/documents`. |\n| Full library (title/author/series/%) from `cc.db` | **Yes** | `cc.db` is in `/var/local` — not exposed over USB; needs root/SSH. |\n| \"Mark as Read\" status (`fmcache.db`) | **Yes** | Under `/mnt/us/system` — needs system access. |\n| Full dashboard with SSH sync | **Yes** | SSH only exists on a jailbroken device. |\n\n### Connecting to the Kindle\n\nTwo ways — both supported:\n\n1. **SSH over USB networking (usbnet)** — default. A jailbroken Kindle with usbnet exposes an SSH server (busybox `dropbear`/`sshd`, **no SFTP**). Connection details come from environment variables (defaults match the standard usbnet setup):\n\n   ```bash\n   export KINDLE_HOST=192.168.15.244   # usbnet device IP (default)\n   export KINDLE_USER=root             # default\n   export KINDLE_PW=kindle             # the well-known default jailbreak password\n   ```\n\n   **Public-key login.** If your device is configured for key-only SSH (public\n   key in `authorized_keys`, password auth disabled), point at a private key and\n   clear the password:\n\n   ```bash\n   export KINDLE_SSH_KEY=~/.ssh/id_ed25519   # private-key path\n   export KINDLE_PW=                         # empty = don't try password\n   ```\n\n   `ssh-agent` and default `~/.ssh` keys are also picked up automatically\n   (`look_for_keys`/`allow_agent` are on). Password stays the default, so\n   existing setups are unaffected. (Reported on MobileRead:\n   [thread #373997](https://www.mobileread.com/forums/showthread.php?t=373997).)\n\n   If Windows shows the Kindle as *\"Unknown USB Device (Device Descriptor Request Failed)\"* or as a COM port instead of a network adapter, see [docs/usb-networking.md](docs/usb-networking.md).\n\n2. **USB mass storage / local folder** — mount the Kindle as a drive and point the tool at the `documents` folder. (`cc.db` is **not** on the USB partition, so pull it once via SSH; sidecars are on the USB partition.)\n\n---\n\n## Usage\n\n```bash\n# SSH sync (cc.db + fmcache.db + all sidecars) -\u003e build -\u003e open browser\npython reader-dashboard/dashboard.py serve\n\n# Reuse the existing local cache (no device needed)\npython reader-dashboard/dashboard.py serve --no-sync\n\n# Read sidecars from a locally-mounted Kindle instead of SSH\npython reader-dashboard/dashboard.py serve --local D:/documents\n\n# Just sync, or just rebuild library.json\npython reader-dashboard/dashboard.py sync\npython reader-dashboard/dashboard.py build\n```\n\nOpens at `http://127.0.0.1:8742/`. The **🔄 Sync** button re-pulls from the device and rebuilds.\n\n### Command-line scripts (no dashboard)\n\nThe `reading-metadata/scripts/` folder has standalone CLI tools:\n\n```bash\npython reading-metadata/scripts/krds.py \u003cfile.azw3f|.azw3r\u003e   # decode one sidecar to JSON\npython reading-metadata/scripts/reading_stats.py D:/documents # per-book time + pace table + CSV\npython reading-metadata/scripts/page_history.py D:/documents  # reading sessions / timeline\npython reading-metadata/scripts/export_annotations.py D:/documents  # highlights/notes -\u003e CSV\npython reading-metadata/scripts/dump_sidecars.py D:/documents # decode every sidecar to JSON\n```\n\n---\n\n## How it works\n\n1. **sync** — pulls `cc.db`, `fmcache.db`, and every `.azw3f`/`.azw3r` under `documents` into `reader-dashboard/cache/`, preserving the folder tree. Over SSH it streams each file with `cat` (busybox has no SFTP) on a single connection, with proper shell-quoting for paths containing spaces and apostrophes.\n2. **build** — joins each `cc.db` `Entry:Item` to its `.sdr` sidecar via `p_location`, decodes the KRDS sidecars with the patched `krds.py`, merges the `fmcache.db` read-state + sessions, computes derived stats, and writes `reader-dashboard/web/library.json`.\n3. **serve** — a stdlib `http.server` serves `web/` (with `no-store` headers) plus a `POST /api/refresh` endpoint.\n\n---\n\n## Repository layout\n\n```\nkindle-reading-dashboard/\n├─ ksh.py                       # tiny SSH run/get/put helper (paramiko, env-configured)\n├─ reader-dashboard/\n│  ├─ dashboard.py              # sync + build + serve (single entrypoint)\n│  └─ web/                      # index.html, book.html, common.js, style.css\n├─ reading-metadata/\n│  └─ scripts/                  # krds.py (patched) + standalone CLI tools\n└─ docs/\n   ├─ KRDS-format.md            # the .azw3f/.azw3r binary format, every structure\n   ├─ cc-db-schema.md           # the cc.db library database schema\n   ├─ read-state-storage.md     # where \"Mark as Read\" actually lives\n   └─ usb-networking.md         # Kindle usbnet + Windows RNDIS troubleshooting\n```\n\nPersonal reading data (`cache/`, `library.json`, CSVs, decoded sidecars) is **git-ignored** and never committed.\n\n---\n\n## Credits\n\n- **`reading-metadata/scripts/krds.py`** — Kindle Reader Data Store parser by **John Howell** (`jhowell@acm.org`), © 2019, **GPL v3**. Background and format history:\n  - https://www.mobileread.com/forums/showthread.php?t=322172\n  - https://github.com/K-R-D-S/KRDS\n  - This repo's copy is **patched for FW 5.18.x** (a `raw` fallback in `decode_object` and draining of unknown trailing fields) so newer firmware structures don't break decoding.\n- **`cc.db` schema** documentation cross-referenced with the [kindlewiki cc.db notes](https://sighery.github.io/kindlewiki/kindle-hacking/cc.html) and the [kindle-series-manager](https://github.com/sighery/kindle-series-manager) project.\n- **Windows RNDIS driver** for Kindle USB networking by **Marco77** (MobileRead): https://www.mobileread.com/forums/showthread.php?p=3283986\n- Everything else (dashboard, fmcache read-state discovery, page mapping, FW-5.18 patches) written for this project.\n\n## License\n\n**GPL v3** — required because it incorporates the GPLv3-licensed `krds.py`. See [LICENSE](LICENSE).\n\n## Disclaimer\n\n- **Not affiliated with Amazon.** \"Kindle\" and \"Amazon\" are trademarks of Amazon.com, Inc. This is an independent, unofficial project; the names are used only descriptively to indicate compatibility (nominative fair use).\n- **For personal use** with your own device and your own books/documents. It reads metadata files the device already wrote; it performs **no writes** to the device (read-only sync).\n- **No DRM circumvention.** This tool reads reading-metadata and library files only. It does **not** decrypt books, strip DRM, or access protected content.\n- **Jailbreak.** The full dashboard assumes a device with USB networking/root, which you enable yourself. Jailbreaking may violate Amazon's Terms of Service; that is between you and Amazon. This software does not jailbreak anything — it only reads files once access exists.\n- Provided **as-is, without warranty**, under the GPL v3. Use at your own risk.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzevisvei%2Fkindle-reading-dashboard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzevisvei%2Fkindle-reading-dashboard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzevisvei%2Fkindle-reading-dashboard/lists"}