{"id":46160953,"url":"https://github.com/aredesrafa/logslimmer","last_synced_at":"2026-03-02T11:02:00.063Z","repository":{"id":324621135,"uuid":"1096496202","full_name":"aredesrafa/logslimmer","owner":"aredesrafa","description":"Save massive token costs while coding or vibe-coding by semantically compressing console and terminal logs with up to 90% reduction, preserving all relevant information for AI agents.","archived":false,"fork":false,"pushed_at":"2025-12-06T00:30:56.000Z","size":2039,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-09T07:52:36.729Z","etag":null,"topics":["ai-tools","cli","compressor","context-engineering","developer-experience","devtools","log","log-processing","observability","prompt-engineering","text-compression","vibe-coding","workers"],"latest_commit_sha":null,"homepage":"https://aredesrafa.github.io/logslimmer/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aredesrafa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-14T14:08:13.000Z","updated_at":"2025-12-05T23:29:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/aredesrafa/logslimmer","commit_stats":null,"previous_names":["aredesrafa/logslimmer"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/aredesrafa/logslimmer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aredesrafa%2Flogslimmer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aredesrafa%2Flogslimmer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aredesrafa%2Flogslimmer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aredesrafa%2Flogslimmer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aredesrafa","download_url":"https://codeload.github.com/aredesrafa/logslimmer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aredesrafa%2Flogslimmer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29999221,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-02T09:59:02.300Z","status":"ssl_error","status_checked_at":"2026-03-02T09:59:02.001Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-tools","cli","compressor","context-engineering","developer-experience","devtools","log","log-processing","observability","prompt-engineering","text-compression","vibe-coding","workers"],"created_at":"2026-03-02T11:01:59.310Z","updated_at":"2026-03-02T11:02:00.052Z","avatar_url":"https://github.com/aredesrafa.png","language":"JavaScript","readme":"# LogSlimmer\n\nToken-first log compression and timeline recaps that keep autonomous agents efficient.\n\n![Svelte](https://img.shields.io/badge/Svelte-4-FF3E00?style=flat-square\u0026logo=svelte)\n![Vite](https://img.shields.io/badge/Vite-5-646CFF?style=flat-square\u0026logo=vite)\n![License](https://img.shields.io/badge/License-MIT-green?style=flat-squar\n[![Demo](https://img.shields.io/badge/Live%20Demo-LogSlimmer-blue)](https://aredesrafa.github.io/logslimmer/)  \n**➡️ Want to help shape LogSlimmer?** [Join the discussions](https://github.com/aredesrafa/logslimmer/discussions) and share your feedback.\n\n⭐ If you find it useful, consider giving the repository a star — it really helps.\ne)\n\nLogSlimmer ships two instant pipelines:\n\n- **LogSlimmer** — compresses raw console or terminal streams while preserving actionable context.\n- **LogRecap** — turns multi-turn agent transcripts into a compact, navigable recap.\n\nBoth engines are purpose-built to **save tokens for downstream agents**. Inputs up to **100k tokens process in milliseconds**, so you can keep iterating without throttling your context window.\n\n## Why LogSlimmer \u0026 LogRecap?\n\n- **Aggressive token savings**: Typical reductions stay above 90% even on noisy logs.\n- **Signal over noise**: Pattern-aware clustering and heuristics surface the actions, errors, and files that matter.\n- **Hands-free workflows**: Paste text, drop a file, or run the CLI — the pipelines auto-clean and dedupe for you.\n- **Consistent speed**: Browser-friendly workers deliver instantaneous feedback for large pastes.\n\n## Quick Start\n\n### Installation\n\n```bash\npnpm install\npnpm dev\n```\n\nOpen `http://localhost:3000` and paste your logs.\n\n### Build\n\n```bash\npnpm build\npnpm preview\n```\n\n## Pipelines\n\n### LogSlimmer (Token Compression)\n- Detects structural patterns, stack traces, diffs, and repetitions automatically.\n- Collapses redundant blocks while keeping representative examples.\n- Counts tokens so you can compare before/after usage instantly.\n\n### LogRecap (Agent Timeline)\n- Parses coding-agent transcripts into a chronological storyline.\n- Tracks files, issues, commands, and decisions for quick playback.\n- Emits Markdown recaps suitable for pasting back into agent chats or storing alongside artifacts.\n\n## Features\n\n- 📊 **Structural clustering** — groups repetitions, diff chunks, and stack traces automatically.\n- 🧠 **Agent-focused recap** — LogRecap rebuilds investigation timelines for long-running sessions.\n- 🎯 **Token accounting** — live counters show original vs. compressed usage.\n- 📁 **Flexible ingestion** — paste, drag \u0026 drop, or call the CLI helpers under `src/cli/`.\n- 🌙 **Themable UI** — light/dark modes with persisted preferences.\n- ⚡ **Instant feedback** — browser workers keep processing interactive even on six-figure token inputs.\n\n## Project Structure\n\n```\nsrc/\n├── App.svelte                 # Main Svelte interface\n├── app.css\n├── cli/                       # CLI entry points (LogSlimmer \u0026 LogRecap)\n├── config.js                  # Shared configuration flags\n├── log-pipeline/              # LogSlimmer compression pipeline\n│   ├── cluster-builder-no-embeddings.js\n│   ├── config-clustering.js\n│   ├── core/\n│   └── …\n├── log-recap/                 # LogRecap timeline pipeline\n│   └── pipeline.js\n├── utils/                     # Tokenizers, caches, shared helpers\n├── worker-logslimmer.js       # Worker for LogSlimmer\n├── worker-logrecap.js         # Worker for LogRecap\n└── main.ts\n```\n\n## Usage Example\n\n**Input:** 120,000 token debugging session\n**LogSlimmer Output:** 10,500 tokens (~91% reduction)\n**LogRecap Output:** Markdown recap listing investigations, fixes, and unresolved issues\n\nPaste your logs and get a clean, compressed version ready for AI agents.\n\n## Performance\n\n| Input Size | LogSlimmer | LogRecap |\n|------------|------------|----------|\n| \u003c 10k tokens | Instant | Instant |\n| 10k–100k tokens | Instant (\u003c100 ms) | Instant (\u003c100 ms) |\n| 100k–200k tokens | \u003c 1 s | \u003c 1 s |\n\n## Settings\n\n### LogSlimmer pipeline (scoring/noise/truncation)\n- Defaults live in `src/log-pipeline/pipeline-config.js`.\n- You can override via env `LOGSLIMMER_CONFIG_JSON='{\"latencyBuckets\":[{\"minMs\":800,\"weight\":2}],\"statusWeights\":{\"5xx\":6}}'`.\n- Or set `globalThis.LOGSLIMMER_CONFIG = { ... }` before running in-browser.\n- Debug scoring reasons: `LOGSLIMMER_DEBUG_SCORE=true`.\n- Dump defaults + resolved config: `npm run dump:config`.\n\nKey fields:\n- `latencyBuckets`: list of `{ minMs, weight, label }` applied cumulatively.\n- `statusWeights`: per-series (`2xx`, `4xx`, `5xx`) and optional per-code (`404`).\n- `messageWeights`: regex → weight; keep it short for performance.\n- `noisePatterns`: regex list to discard early (health/heartbeat/etc.).\n- `maxLineLength`: clamp long lines when rendering.\n- `showOtherInSummary`: include/exclude category `Other` in summary.\n- `scoreCutoffNonOther` / `scoreCutoffOther`: minimum score to keep events.\n- `maxOtherClusters` / `maxClusters`: limits for rendered clusters.\n- `miscUniqueLimit`: how many single-occurrence events to show in “Unique Events”.\n- `keepFileLinePrefix`: keep `file.ts:37` or `GET /path 500` prefixes intact even when truncating payloads.\n- `preserveTimestamps`: avoid normalizing timestamps away.\n- `keepHumanNotes`: boost/keep free-text narrative lines.\n- `stackFramePreviewHead` / `stackFramePreviewTail`: how many frames to show when collapsing stack traces.\n- Scoring tweaks (defaults): 2xx gets extra negative weight; repetitive CORS/claim success patterns are down-weighted; “Unique Events” only shows error/4xx/5xx/aborted/timeout or very high latency.\n\n## Dependencies\n\n- **js-tiktoken** - Token counting\n- **Svelte** - UI framework\n- **Vite** - Build tool\n- **Tailwind CSS** - Styling\n\n## License\n\nMIT - See LICENSE file\n\n## Support\n\nFound an issue? Open a [GitHub issue](../../issues)\n\n---\n\n**LogSlimmer** - Stop wasting tokens on redundant logs. Share smart summaries with your AI agents. 🚀\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faredesrafa%2Flogslimmer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faredesrafa%2Flogslimmer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faredesrafa%2Flogslimmer/lists"}