{"id":51457821,"url":"https://github.com/shw1606/runscroll","last_synced_at":"2026-07-06T02:01:38.263Z","repository":{"id":360710215,"uuid":"1231993027","full_name":"shw1606/runscroll","owner":"shw1606","description":"Single-file scrollable HTML report for one-shot batch jobs (ETL, ML training, migrations, crawlers). Streaming append-write. No server. No infra.","archived":false,"fork":false,"pushed_at":"2026-05-27T14:07:13.000Z","size":508,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-27T16:08:07.139Z","etag":null,"topics":["batch","crawler","data-quality","etl","html","logging","ml-training","mlops","pipeline","post-mortem","python","report","single-file","streaming","visualization"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/runscroll/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shw1606.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-05-07T13:37:53.000Z","updated_at":"2026-05-27T14:10:13.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/shw1606/runscroll","commit_stats":null,"previous_names":["shw1606/runscroll"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/shw1606/runscroll","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shw1606%2Frunscroll","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shw1606%2Frunscroll/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shw1606%2Frunscroll/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shw1606%2Frunscroll/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shw1606","download_url":"https://codeload.github.com/shw1606/runscroll/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shw1606%2Frunscroll/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35175119,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-06T02:00:07.184Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch","crawler","data-quality","etl","html","logging","ml-training","mlops","pipeline","post-mortem","python","report","single-file","streaming","visualization"],"created_at":"2026-07-06T02:01:35.311Z","updated_at":"2026-07-06T02:01:38.198Z","avatar_url":"https://github.com/shw1606.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# runscroll\n\n[![PyPI](https://img.shields.io/pypi/v/runscroll.svg)](https://pypi.org/project/runscroll/)\n[![Python versions](https://img.shields.io/pypi/pyversions/runscroll.svg)](https://pypi.org/project/runscroll/)\n[![License: MIT](https://img.shields.io/pypi/l/runscroll.svg)](https://github.com/shw1606/runscroll/blob/main/LICENSE)\n\n\u003e **runscroll — turn one batch run into one scrollable HTML report.**\n\u003e\n\u003e Sprinkle `report.add_*()` calls through your batch job. Get a single\n\u003e self-contained HTML file out the other side. Mail it, drop it in S3,\n\u003e attach it to a PR. No server. No account. No infrastructure.\n\n![runscroll rendered ML training run example](https://raw.githubusercontent.com/shw1606/runscroll/main/images/runscroll-ml.png)\n\n## When to use this\n\n| Your situation                                          | runscroll? |\n| ------------------------------------------------------- | ---------- |\n| Batch job ran, want to share what happened              | ✅          |\n| Daily ETL done, mail a result page to oncall            | ✅          |\n| ML training run done, drop a post-mortem in the PR      | ✅          |\n| Migration finished, link an audit page from a ticket    | ✅          |\n| Crawler finished, browse failures in one HTML           | ✅          |\n| Live monitoring dashboard                               | ❌ Grafana / Datadog |\n| Compare 50 experiment runs                              | ❌ MLflow / Weights \u0026 Biases |\n| Interactive notebook for exploration                    | ❌ Jupyter |\n| Real-time streaming logs                                | ❌ stdlib `logging` |\n| Generic HTML page builder                               | ❌ `dominate` / `yattag` |\n\nIf your row above says \"❌\", that other tool is the right fit — runscroll is\nintentionally narrow.\n\n## Install\n\n```bash\npip install runscroll                                # core, stdlib only\npip install \"runscroll[matplotlib,plotly,pil]\"       # with adapters\n```\n\n## 30-second example\n\n```python\nfrom runscroll import Collector\n\nwith Collector(\"report.html\", title=\"Daily ETL\") as report:\n    report.add_kv({\"started_at\": \"2026-05-05T09:00\", \"config\": \"v17\"})\n\n    with report.section(\"Extract\"):\n        report.add_text(f\"loaded {len(rows):,} rows\")\n        report.add_table(rows[:5], title=\"Sample input\")\n\n    with report.section(\"Transform\"):\n        report.add_text(\"dropped 142 rows (0.3%)\", level=\"warning\")\n        report.add_table(dropped[:20], title=\"Sample dropped rows\")\n\n    report.add_text(\"done\", level=\"success\")\n```\n\nThat produces `report.html` — one file, no assets folder, no external CDN.\nOpen it in any browser, mail it, upload it to S3, attach it to a PR.\n\n## API surface (the entire library)\n\n```python\nCollector(path, title, mode=\"inline\"|\"directory\", asset_writer=None, log_exceptions=True)\n    # context manager: with Collector(...) as report: ...\n\nreport.add_text(text, level=\"info\"|\"debug\"|\"warning\"|\"error\"|\"success\")\nreport.add_kv(mapping, title=\"\")\nreport.add_code(code, lang=\"\", title=\"\")\nreport.add_table(list_of_dicts_or_lists, title=\"\")\nreport.add_image(bytes_or_path_or_PIL_or_ndarray, caption=\"\", title=\"\")\nreport.add_figure(matplotlib_or_plotly_figure, title=\"\", description=\"\", close=True)\n\nwith report.section(name):              # nested allowed\n    ...\n```\n\nThat's it. The whole library is one class with eight methods.\n\n## Output modes\n\n```python\n# inline (default) — one .html file, all assets base64'd in\nCollector(\"report.html\", mode=\"inline\")\n\n# directory — index.html + assets/ folder; works as a static site\nCollector(\"report/\", mode=\"directory\")\n\n# directory + custom destination — plug in S3 / GCS via AssetWriter\nCollector(\"report/\", mode=\"directory\", asset_writer=MyS3Writer(...))\n```\n\nThe `AssetWriter` protocol is one method:\n\n```python\nclass AssetWriter(Protocol):\n    def write(self, relative_path: str, content: bytes) -\u003e None: ...\n```\n\nThat's all the library asks. Authentication, region, retries, caching are\nyour concern — runscroll never imports a cloud SDK.\n\n## Recipes\n\nWorking scripts in `examples/` — drop them next to your pipeline as a\nstarting point.\n\n### ML training run — [examples/ml_training_run.py](examples/ml_training_run.py)\n\n![ML training run screenshot](https://raw.githubusercontent.com/shw1606/runscroll/main/images/runscroll-ml.png)\n\nLoss curves, confusion matrix, per-class precision/recall, sample worst\npredictions as inline images. Exercises matplotlib + numpy + PIL + nested\nsections.\n\n```python\nwith Collector(out, title=f\"Train run {run_id}\") as report:\n    report.add_kv({\"model\": \"resnet50\", \"lr\": 3e-4, \"bs\": 64, \"seed\": 42})\n    with report.section(\"Training\"):\n        for epoch in range(epochs):\n            report.add_text(f\"epoch={epoch}  train={tl:.4f}  val={vl:.4f}\")\n        report.add_figure(plot_loss_curves(history), title=\"Loss curves\")\n    with report.section(\"Holdout\"):\n        report.add_figure(plot_confusion(y_true, y_pred), title=\"Confusion\")\n        report.add_table(per_class_metrics, title=\"Per-class metrics\")\n```\n\n### Daily ETL — [examples/data_quality_etl.py](examples/data_quality_etl.py)\n\n![Daily ETL screenshot](https://raw.githubusercontent.com/shw1606/runscroll/main/images/runscroll-etl.png)\n\nHourly volume, drop-rate warning with a sample of dropped rows, post-clean\ndistribution. The single-file output ships in a mail attachment.\n\n### Migration validation — [examples/migration_validation.py](examples/migration_validation.py)\n\n![Migration validation screenshot](https://raw.githubusercontent.com/shw1606/runscroll/main/images/runscroll-migration.png)\n\nPer-table validation with **interactive plotly distributions** — zoom,\npan, hover tooltips, all in the single self-contained file. The plotly\nbundle is inlined exactly once even when there are dozens of figures.\nThe `6 warnings` badge in the top-right corner is generated client-side\nby counting `rs-text-warning` entries.\n\n### Web crawler — [examples/web_scraper.py](examples/web_scraper.py)\n\n![Web crawler screenshot](https://raw.githubusercontent.com/shw1606/runscroll/main/images/runscroll-crawler.png)\n\nStatus-code breakdown, per-request latency histogram, every failed URL\nin a browsable table.\n\n## How memory stays flat\n\nEach `add_*` call serializes its content to disk and flushes immediately.\n**There is no in-memory entry buffer.** A 500 MiB report uses the same\nRAM as a 5 KiB one — only a counter, a section-depth integer, and a file\nhandle live in Python.\n\nThis is the design's first-priority guarantee. The test\n`tests/test_streaming_memory.py` keeps it honest: 30 × 10 MiB writes must\nleave less than `total_written / 30` resident, and a 30 MiB on-disk image\nstreamed through `add_image` must not grow RSS by more than 1 MiB.\n\n## What this library is NOT\n\n- ❌ A live monitoring dashboard — Grafana / Datadog.\n- ❌ A multi-run experiment tracker — MLflow / Weights \u0026 Biases.\n- ❌ An interactive notebook — Jupyter.\n- ❌ A general HTML builder — `dominate` / `yattag`.\n- ❌ A cloud SDK wrapper — supply your own `AssetWriter`.\n- ❌ A static site generator — Sphinx / mkdocs.\n\n## License\n\nMIT.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshw1606%2Frunscroll","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshw1606%2Frunscroll","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshw1606%2Frunscroll/lists"}