{"id":44585487,"url":"https://github.com/mlorentedev/pollex","last_synced_at":"2026-03-05T06:02:36.625Z","repository":{"id":337749959,"uuid":"1155051841","full_name":"mlorentedev/pollex","owner":"mlorentedev","description":"Text polishing API (Go) + Chrome extension + llama.cpp GPU inference on Jetson Nano. Self-hosted, private, fast.","archived":false,"fork":false,"pushed_at":"2026-02-18T03:03:38.000Z","size":976,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-02-18T07:39:39.312Z","etag":null,"topics":["chrome-extension","go","gpu-inference","jetson-nano","llama-cpp","llm","self-hosted","text-processing"],"latest_commit_sha":null,"homepage":"https://github.com/mlorentedev/pollex","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mlorentedev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-11T04:15:11.000Z","updated_at":"2026-02-18T03:03:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mlorentedev/pollex","commit_stats":null,"previous_names":["mlorentedev/pollex"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/mlorentedev/pollex","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlorentedev%2Fpollex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlorentedev%2Fpollex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlorentedev%2Fpollex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlorentedev%2Fpollex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mlorentedev","download_url":"https://codeload.github.com/mlorentedev/pollex/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mlorentedev%2Fpollex/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30111779,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-05T03:40:26.266Z","status":"ssl_error","status_checked_at":"2026-03-05T03:39:15.902Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chrome-extension","go","gpu-inference","jetson-nano","llama-cpp","llm","self-hosted","text-processing"],"created_at":"2026-02-14T06:03:43.356Z","updated_at":"2026-03-05T06:02:36.607Z","avatar_url":"https://github.com/mlorentedev.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pollex\n\n**Polish your English text** — fixes grammar, improves coherence, and tightens wording. The output sounds like a fluent non-native speaker: professional and clear, not AI-generated.\n\nSelf-hosted, private, and fast. Runs entirely on a Jetson Nano 4GB with GPU inference via llama.cpp.\n\n![Pollex demo](docs/assets/demo.gif)\n\n## Architecture\n\n```mermaid\ngraph LR\n    subgraph Your Machine\n        EXT[\"Browser Extension\u003cbr/\u003e(Manifest V3)\"]\n    end\n\n    subgraph Internet\n        CF[\"Cloudflare Tunnel\u003cbr/\u003e(pollex.mlorente.dev)\"]\n    end\n\n    subgraph Jetson Nano 4GB\n        API[\"Pollex API\u003cbr/\u003e(Go · :8090)\"]\n        LLAMA[\"llama-server\u003cbr/\u003e(CUDA 10.2 · GPU)\"]\n        MODEL[\"Qwen 2.5 1.5B\u003cbr/\u003e(Q4_0 · ~1GB)\"]\n    end\n\n    subgraph Monitoring\n        PROM[\"Prometheus + Grafana\"]\n    end\n\n    EXT -- \"HTTPS + API Key\" --\u003e CF\n    CF -- \"localhost:8090\" --\u003e API\n    API -- \"/v1/chat/completions\" --\u003e LLAMA\n    LLAMA --\u003e MODEL\n    PROM -. \"/metrics\" .-\u003e CF\n\n    style EXT fill:#4a90d9,stroke:#3a7bc8,color:#fff\n    style CF fill:#f48120,stroke:#d35400,color:#fff\n    style API fill:#2ecc71,stroke:#27ae60,color:#fff\n    style LLAMA fill:#e67e22,stroke:#d35400,color:#fff\n    style MODEL fill:#f39c12,stroke:#e67e22,color:#fff\n    style PROM fill:#9b59b6,stroke:#8e44ad,color:#fff\n```\n\n| Layer | Tech | Role |\n| --- | --- | --- |\n| Extension | Chrome Manifest V3 | UI — paste text, select model, copy result |\n| Tunnel | Cloudflare Tunnel | Zero-config ingress (Jetson behind double NAT) |\n| API | Go 1.26, stdlib `net/http` | Routes text to LLM backends, returns polished result |\n| LLM | llama.cpp + Qwen 2.5 1.5B Q4_0 | Local GPU inference on Jetson Nano (~3s short, ~16s medium) |\n| Monitoring | Prometheus + Alertmanager + Grafana | SLO tracking, alerting, dashboards |\n\n## How It Works\n\n```mermaid\nsequenceDiagram\n    participant U as User\n    participant E as Extension\n    participant T as Cloudflare Tunnel\n    participant A as Pollex API\n    participant L as llama-server\n\n    U-\u003e\u003eE: Paste text + click Polish\n    E-\u003e\u003eE: Show spinner (0s...)\n    E-\u003e\u003eT: POST /api/polish (X-API-Key)\n    T-\u003e\u003eA: Forward to localhost:8090\n    A-\u003e\u003eL: POST /v1/chat/completions\n    L-\u003e\u003eL: GPU inference (~3-8s)\n    L--\u003e\u003eA: Polished text\n    A--\u003e\u003eT: {\"polished\":\"...\", \"elapsed_ms\":...}\n    T--\u003e\u003eE: Response\n    E-\u003e\u003eE: Hide spinner, show result\n    U-\u003e\u003eE: Click Copy\n```\n\n## Quick Start\n\n### Development (no GPU needed)\n\n```sh\nmake dev    # Start API with mock adapter on :8090\nmake test   # Run all tests (80+ with subtests, race detector)\n```\n\nLoad the extension: `chrome://extensions` → Developer mode → Load unpacked → select `extension/`.\n\nRun `make help` for the full list of targets (35 total: dev, build, bench, docker, monitoring, deploy, loadtest, jetson remote ops).\n\n## Performance (Jetson Nano 4GB)\n\nMeasured on Qwen 2.5 1.5B Q4_0, full GPU offload (`-ngl 999`), 128 Maxwell cores.\n\n| Text Length | Chars | Inference Time | Throughput |\n| ----------- | ----- | -------------- | ---------- |\n| Short | ~50 | ~3s | ~4 tok/s |\n| Medium | ~500 | ~8s | ~4 tok/s |\n| Long | ~1000 | ~16s | ~4 tok/s |\n\n**SLO targets (7-day rolling window):**\n\n| SLI | Target |\n| --- | --- |\n| Availability (API + llama-server) | 99% (≤100.8 min downtime) |\n| Latency p50 | \u003c 20s |\n| Latency p95 | \u003c 60s |\n| Error rate (5xx on `/api/polish`) | \u003c 1% |\n\nLoad tested with k6 (burst: 5 VUs / 25 iterations, sustained: 12 req/min for 2 min). Alerting via Prometheus rules + Alertmanager.\n\n## API\n\n| Method | Path | Auth | Description |\n| --------- | ----------- | ----------- | ----------------------- |\n| `POST` | `/api/polish` | `X-API-Key` | Polish text via selected model |\n| `GET` | `/api/models` | `X-API-Key` | List available models |\n| `GET` | `/api/health` | None | Health check (per-adapter status) |\n| `GET` | `/metrics` | None | Prometheus metrics |\n\n### `POST /api/polish`\n\n```sh\ncurl -X POST https://pollex.mlorente.dev/api/polish \\\n  -H 'Content-Type: application/json' \\\n  -H 'X-API-Key: YOUR_KEY' \\\n  -d '{\"text\":\"i goes to store yesterday\",\"model_id\":\"qwen2.5-1.5b-gpu\"}'\n\n# {\"polished\":\"I went to the store yesterday.\",\"model\":\"qwen2.5-1.5b-gpu\",\"elapsed_ms\":3200}\n```\n\n### `GET /api/health`\n\n```json\n{\n  \"status\": \"ok\",\n  \"version\": \"1.4.0\",\n  \"adapters\": {\n    \"qwen2.5-1.5b-gpu\": {\"available\": true},\n    \"claude-sonnet\": {\"available\": false, \"reason\": \"no API key\"}\n  }\n}\n```\n\n### `GET /api/models`\n\n```json\n[\n  {\"id\": \"qwen2.5-1.5b-gpu\", \"name\": \"Qwen 2.5 1.5B (GPU)\", \"provider\": \"llama.cpp\"},\n  {\"id\": \"mock\", \"name\": \"Mock\", \"provider\": \"mock\"}\n]\n```\n\n## Project Structure\n\n```text\npollex/\n├── cmd/\n│   ├── pollex/              # Entry point (flags, config, wiring, shutdown)\n│   └── benchmark/           # Benchmark CLI tool\n├── internal/\n│   ├── adapter/             # LLMAdapter interface + implementations\n│   │   ├── adapter.go       #   Interface: Name(), Polish(), Available()\n│   │   ├── mock.go          #   Mock (dev/testing)\n│   │   ├── ollama.go        #   Ollama (legacy, optional)\n│   │   ├── claude.go        #   Claude API (optional)\n│   │   └── llamacpp.go      #   llama.cpp (primary, GPU)\n│   ├── config/              # YAML + env overrides (POLLEX_*)\n│   ├── handler/             # HTTP handlers + response helpers\n│   ├── metrics/             # Prometheus metric declarations (promauto)\n│   ├── middleware/           # CORS, RequestID, Logging, Metrics, APIKey, RateLimit, MaxBytes\n│   └── server/              # SetupMux + integration tests\n├── extension/               # Chrome extension (Manifest V3)\n├── prompts/polish.txt       # System prompt\n├── deploy/\n│   ├── loadtest/            # k6 load test scripts (normal, burst, jetson, soak)\n│   ├── systemd/             # pollex-api, llama-server, cloudflared, jetson-clocks services\n│   ├── scripts/             # init, build-llamacpp, setup-cloudflared\n│   ├── prometheus/          # Alert rules, scrape config, alertmanager\n│   ├── grafana/             # Dashboard JSON + provisioning\n│   └── config.yaml          # Production config (deployed to Jetson)\n├── Dockerfile               # Multi-stage: Go builder → alpine (24.7MB)\n├── docker-compose.yml       # Local dev (mock mode)\n├── docker-compose.monitoring.yml  # Prometheus + Alertmanager + Grafana\n├── .github/workflows/       # CI (lint+test+build) + Release (goreleaser)\n└── Makefile\n```\n\n## Contributing\n\n### Prerequisites\n\n- Go 1.26+\n- Chrome (for extension testing)\n\n### Development Workflow\n\n1. **Run tests first** to ensure a clean baseline:\n\n   ```sh\n   make test\n   make lint\n   ```\n\n2. **Start the dev server** with the mock adapter (no LLM needed):\n\n   ```sh\n   make dev\n   ```\n\n3. **Load the extension** in Chrome (`chrome://extensions` → Load unpacked → `extension/`).\n\n4. **Make changes** — the adapter pattern makes it easy to add new LLM backends:\n   - Implement the `LLMAdapter` interface in `internal/adapter/`\n   - Register it in `cmd/pollex/main.go:buildAdapters()`\n   - The rest (routing, health checks, model listing) is automatic\n\n5. **Run tests** before pushing:\n\n   ```sh\n   make test   # All tests with race detector\n   make lint   # go vet + gofmt\n   ```\n\n### Middleware Chain\n\nRequest processing order (defined in `internal/middleware/chain.go`):\n\n```text\nCORS → RequestID → Logging → Metrics → APIKey → RateLimit → MaxBytes(64KB) → Timeout(120s) → Router\n```\n\n### Hardening\n\n| Protection | Limit | Response |\n| --- | --- | --- |\n| API key | `X-API-Key` header, constant-time compare | 401 |\n| Request body | 64KB max | 413 |\n| Text length | 10,000 chars | 400 |\n| Rate limit | 10 req/min/IP (sliding window) | 429 |\n| Request timeout | 120s | 504 |\n\n### CI/CD\n\n- **Push to `master`** or **PR** → lint + test + build (amd64 + arm64)\n- **Tag `v*`** → goreleaser creates GitHub release with binaries + extension zip\n\nCommit messages follow [Conventional Commits](https://www.conventionalcommits.org/).\n\n## Docker\n\n```sh\nmake docker-build   # Build image (alpine:3.21, 24.7MB, non-root)\nmake docker-dev     # Run pollex in Docker (mock mode, :8090)\nmake docker-down    # Stop container\n```\n\n### Monitoring Stack\n\n```sh\nmake dev              # Start pollex natively (mock mode)\nmake monitoring-up    # Start Prometheus + Alertmanager + Grafana\n```\n\n- Prometheus: [localhost:9090](http://localhost:9090) — 6 alerting rules based on SLOs\n- Grafana: [localhost:3000](http://localhost:3000) — Pollex SRE Overview dashboard (auto-provisioned)\n- Alertmanager: [localhost:9093](http://localhost:9093) — Slack webhook routing\n\n```sh\nmake monitoring-down      # Stop monitoring stack\nmake monitoring-validate  # Validate Prometheus rules syntax\n```\n\n## Deploy to Jetson\n\n### First-time setup\n\n```sh\nmake deploy-init      # Packages, CUDA PATH, /etc/pollex, systemd services\nmake deploy-llamacpp  # Build llama.cpp with CUDA on Jetson (~85 min)\nmake deploy           # Binary + config + prompt\nmake deploy-secrets   # API key\nmake deploy-tunnel    # Cloudflare Tunnel\n```\n\n### Subsequent deploys\n\n```sh\nmake deploy           # Build ARM64 + SCP + restart service\n```\n\n### Remote operations\n\n```sh\nmake jetson-ssh             # SSH into Jetson\nmake jetson-status          # Health check via SSH\nmake jetson-test            # End-to-end polish test\nmake jetson-logs            # Tail API logs\nmake jetson-tunnel-start    # Start Cloudflare Tunnel\nmake jetson-tunnel-status   # Tunnel health\nmake jetson-tunnel-logs     # Tail tunnel logs\n```\n\n## Hardware\n\n**Jetson Nano 4GB** — ARM64, CUDA 10.2, 128 Maxwell cores.\n\n| Component | RAM |\n| --- | --- |\n| JetPack OS (headless) | ~500MB |\n| llama-server (GPU) | ~200MB |\n| Qwen 2.5 1.5B (Q4) | ~1.0GB |\n| Pollex API | ~15MB |\n| **Free** | **~2.3GB** |\n\n## License\n\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlorentedev%2Fpollex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmlorentedev%2Fpollex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmlorentedev%2Fpollex/lists"}