{"id":47140636,"url":"https://github.com/simbo1905/kapture-dom-erode","last_synced_at":"2026-03-12T22:02:13.742Z","repository":{"id":342586855,"uuid":"1174459233","full_name":"simbo1905/kapture-dom-erode","owner":"simbo1905","description":"Read the *visible* text from kapture-dom snapshots without drowning in `\u003cdiv\u003e` soup.","archived":false,"fork":false,"pushed_at":"2026-03-06T13:59:26.000Z","size":51,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-06T17:31:34.471Z","etag":null,"topics":["agent-skills","claude-code-skill","claude-skills","dom-parsing","kapture","kapture-dom","kapture-mcp","opencode-skill"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simbo1905.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-06T13:17:55.000Z","updated_at":"2026-03-06T13:59:29.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/simbo1905/kapture-dom-erode","commit_stats":null,"previous_names":["simbo1905/kapture-dom-erode"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/simbo1905/kapture-dom-erode","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simbo1905%2Fkapture-dom-erode","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simbo1905%2Fkapture-dom-erode/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simbo1905%2Fkapture-dom-erode/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simbo1905%2Fkapture-dom-erode/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simbo1905","download_url":"https://codeload.github.com/simbo1905/kapture-dom-erode/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simbo1905%2Fkapture-dom-erode/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30446425,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-12T21:31:01.033Z","status":"ssl_error","status_checked_at":"2026-03-12T21:30:43.161Z","response_time":114,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-skills","claude-code-skill","claude-skills","dom-parsing","kapture","kapture-dom","kapture-mcp","opencode-skill"],"created_at":"2026-03-12T22:02:08.674Z","updated_at":"2026-03-12T22:02:13.721Z","avatar_url":"https://github.com/simbo1905.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kapture-dom-erode\n\nYou asked your LLM to check a web page. It saved a DOM dump that blows up its context window. 😱 \n\n```bash\n❯ ls -alh kapture-dom-page-save.html\n-rw-r--r--  1 yourname  staff   1.8M Mar  6 13:06 kapture-dom-page-save.html 😤 \n```\n\nWhat now? Use the kapture-dom-erode skill. Read the *visible* text from multi-megabyte Kapture snapshots without drowning in `\u003cdiv\u003e` soup.\n\n## You Know That Moment When...\n\nYour agent says *\"I can see the page\"* but what it actually has is millions of characters of nested HTML.\n\n**You** see text on screen that is not far apart. The **LLM** sees that text a mile apart inside 47 wrapper tags.\n\n## The Problem\n\n- 🤯 **Kapture DOM dumps are massive** -- 1-5MB of framework-generated tag soup\n- 🔍 **Text is buried** -- \"Bake at 180°C\" lives at `html[0].body[0].div[6].div[2].div[1].div[4].div[0].p[0]`\n- 💸 **Context windows cry** -- feeding raw HTML to your LLM is not ideal\n- ⚠️ **Screenshots aren't parseable** -- you can see it, but the agent can't read it\n\n## The Solution\n\nFour commands. Find what you need. Erode the tags. Done.\n\n```bash\n# Find WHERE text lives in the DOM\n./tools.sh gron-grep -f page.html -q \"Bake at 180\"\n\n# Auto-extract the page's main body text in one shot\n./tools.sh main-text -f page.html\n\n# Rank all block elements by size, auto-detect main content region\n./tools.sh top-content -f page.html\n\n# Strip tags below a known path, return visible text\n./tools.sh extract-text -f page.html -p \"html[0].body[0].div[3]\"\n```\n\n---\n\n## 🚀 Commands\n\n### `top-content` — Rank blocks and detect content region\n\nScans every block element, counts visible characters, prints the top N by size with previews. Also auto-detects the most likely \"real content\" region by spotting the heading + body alternating pattern.\n\n```bash\n./tools.sh top-content -f page.html\n```\n\n```\nPath                                    Chars   Words  Preview\n---------------------------------------------------------------\nhtml[0].body[0].main[0]                18432    2901  Roasted Vegetable Tart Serves 4 Ready in 55 minutes...\nhtml[0].body[0].main[0].div[0]         18201    2870  Roasted Vegetable Tart Serves 4 Ready in 55 minutes...\nhtml[0].body[0].main[0].div[0].div[1]   9823    1544  Ingredients 2 sheets shortcrust pastry 3 courgettes...\n\n--- Content region detection ---\nBest candidate: html[0].body[0].main[0].div[0].div[1]\n  Score: 14  (headings=4, blocks=6)\n```\n\nOptions: `--top N` (default 10), `--no-detect` (skip auto-detection).\n\n### `gron-grep` — Structural grep\n\nSearch any text string and get back its exact coordinates in the HTML tree.\n\n```bash\n./tools.sh gron-grep -f page.html -q \"On Sale\"\n# html[0].body[0].div[3].div[1].div[2].span[0] = \"On Sale\"\n# html[0].body[0].div[3].div[5].div[2].span[0] = \"On Sale\"\n```\n\nOptions: `-q \u003ctext\u003e`, `-i` (case-insensitive).\n\n### `extract-text` — Tag erosion\n\nStrip away all markup and return visible text from any subtree. Reading order preserved.\n\n```bash\n./tools.sh extract-text -f page.html -p \"html[0].body[0].div[3]\"\n# On Sale\n# Wireless Headphones -- Was $99, now $49\n# USB-C Hub -- Was $45, now $29\n```\n\n### `main-text` — One-shot main body extraction\n\nThe fastest path to clean page text. Finds the two largest content blocks, computes their common ancestor, and erodes everything from that ancestor. Because the two biggest text blocks are almost always in the main body — their common ancestor joins all the siblings together, capturing everything above, between, and below them while excluding nav chrome.\n\n```bash\n./tools.sh main-text -f page.html\n```\n\nOutput (diagnostic lines + extracted text):\n```\nTop-1: html[0].body[0].main[0].div[0].section[1]\nTop-2: html[0].body[0].main[0].div[0].section[2]\nCommon ancestor: html[0].body[0].main[0].div[0]\n\nRoasted Vegetable Tart\nServes 4 · Ready in 55 minutes\n\nIngredients\n2 sheets shortcrust pastry\n3 courgettes, sliced\n...\n```\n\nUse `--quiet` / `-q` to suppress the diagnostic lines and get only the text.\n\n\n---\n\n## Example: \"Show Me All The Deals\"\n\nYou see two products with \"Limited Time Deal\" badges. You want all of them.\n\n```bash\n# Find both\n./tools.sh gron-grep -f shop.html -q \"Limited Time Deal\"\n# html[0].body[0].div[2].div[1].div[0].div[3].span[0]\n# html[0].body[0].div[2].div[3].div[0].div[3].span[0]\n\n# Both share ancestor div[2] — extract everything from there\n./tools.sh extract-text -f shop.html -p \"html[0].body[0].div[2]\"\n```\n\nOr skip the manual steps entirely:\n\n```bash\n./tools.sh main-text -f shop.html\n```\n\n---\n\n## Input Format\n\n`-f` accepts either:\n1. The full JSON object returned by `kapture_dom` (recommended) — the tool reads the `\"html\"` key by default\n2. A raw `.html` file\n\nOverride the JSON key with `-k / --content-key` if your dump uses a different field name.\n\n---\n\n## 📖 Quick Reference\n\n```\nSYNOPSIS\n  ./tools.sh main-text    -f \u003cfile\u003e [-q]\n  ./tools.sh top-content  -f \u003cfile\u003e [--top N] [--no-detect]\n  ./tools.sh gron-grep    -f \u003cfile\u003e -q \u003ctext\u003e [-i]\n  ./tools.sh extract-text -f \u003cfile\u003e -p \u003cpath\u003e\n\nOPTIONS (all commands)\n  -f, --file         Input file (kapture_dom JSON or raw HTML)\n  -k, --content-key  JSON key for DOM content (default: html)\n\nmain-text\n  -q, --quiet        Output extracted text only, no diagnostics\n\ntop-content\n  --top N            Show top N results (default: 10)\n  --no-detect        Skip content region auto-detection\n\ngron-grep\n  -q, --query        Text to search for\n  -i                 Case-insensitive search\n\nextract-text\n  -p, --path         Gron-style path to extract from\n```\n\n---\n\n## ⚡ Installation\n\n```bash\n# OpenCode\ncp -r kapture-dom-erode ~/.config/opencode/skills/\n\n# Claude Code\ncp -r kapture-dom-erode ~/.claude/skills/\n```\n\nDependencies: Python + BeautifulSoup, handled via `uv`.\n\n---\n\n## 📜 License\n\nMIT — Copyright (c) 2026 Simon Massey\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimbo1905%2Fkapture-dom-erode","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimbo1905%2Fkapture-dom-erode","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimbo1905%2Fkapture-dom-erode/lists"}