{"id":47667789,"url":"https://github.com/anatolykoptev/go-stealth","last_synced_at":"2026-07-27T05:00:25.188Z","repository":{"id":339738254,"uuid":"1163154513","full_name":"anatolykoptev/go-stealth","owner":"anatolykoptev","description":"Generic anti-ban toolkit for Go: TLS fingerprinting, proxy rotation, rate limiting, generic pool","archived":false,"fork":false,"pushed_at":"2026-07-26T02:41:53.000Z","size":458,"stargazers_count":2,"open_issues_count":4,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-26T04:12:07.107Z","etag":null,"topics":["anti-detection","go","golang","proxy","rate-limiting","scraping","stealth","tls-fingerprint"],"latest_commit_sha":null,"homepage":"https://hully.one","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anatolykoptev.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"docs/roadmap.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-21T07:09:23.000Z","updated_at":"2026-07-26T02:41:48.000Z","dependencies_parsed_at":"2026-03-31T21:02:10.595Z","dependency_job_id":null,"html_url":"https://github.com/anatolykoptev/go-stealth","commit_stats":null,"previous_names":["anatolykoptev/go-stealth"],"tags_count":36,"template":false,"template_full_name":null,"purl":"pkg:github/anatolykoptev/go-stealth","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anatolykoptev%2Fgo-stealth","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anatolykoptev%2Fgo-stealth/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anatolykoptev%2Fgo-stealth/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anatolykoptev%2Fgo-stealth/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anatolykoptev","download_url":"https://codeload.github.com/anatolykoptev/go-stealth/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anatolykoptev%2Fgo-stealth/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35937479,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-27T02:00:06.776Z","response_time":101,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anti-detection","go","golang","proxy","rate-limiting","scraping","stealth","tls-fingerprint"],"created_at":"2026-04-02T12:03:32.336Z","updated_at":"2026-07-27T05:00:25.175Z","avatar_url":"https://github.com/anatolykoptev.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-stealth\n\n[![Go 1.26+](https://img.shields.io/badge/Go-1.26+-00ADD8?logo=go)](go.mod)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n\nGeneric anti-ban toolkit for Go — TLS fingerprinting, proxy rotation, rate limiting, middleware, session management, and a generic pool with health tracking.\n\n**Not a scraping framework** — a reusable HTTP layer that makes any Go HTTP client look like a real browser.\n\n## Features\n\n- **TLS Fingerprinting** — 18 browser profiles (Chrome, Firefox, Safari, Edge) across 5 OS via [tls-client](https://github.com/bogdanfinn/tls-client)\n- **Proxy Rotation** — static list or [Webshare](https://www.webshare.io/) API, per-proxy health tracking with auto-skip\n- **Rate Limiting** — per-key sliding window + per-domain limiter with wildcard matching\n- **Middleware** — composable Handler/Middleware/Chain pattern (logging, retry, rate limit, client hints)\n- **Retry \u0026 Backoff** — exponential backoff with jitter, retryable error detection, generic `RetryDo[T]()`\n- **Generic Pool** — `Pool[T Identity]` with round-robin, health tracking, soft/permanent deactivation, cooldown\n- **Sessions** — fixed profile + cookie jar, request counting, file-based persistence\n- **`http.RoundTripper`** — drop-in replacement for `http.DefaultTransport`\n\n## Install\n\n```bash\ngo get github.com/anatolykoptev/go-stealth\n```\n\n## Quick Start\n\n```go\nclient, _ := stealth.NewClient(\n    stealth.WithProfile(stealth.RandomProfile()),\n)\nbody, headers, status, err := client.Do(\"GET\", \"https://example.com\", nil, nil)\n```\n\n### With Proxy Pool\n\n```go\npool, _ := proxypool.NewWebshare(os.Getenv(\"WEBSHARE_API_KEY\"))\nclient, _ := stealth.NewClient(\n    stealth.WithProxyPool(pool),\n    stealth.WithRetryOnBlock(2),\n)\n```\n\n### Country Targeting\n\nBy default `NewWebshare` targets the US. Use `WebshareConfig` or the rotating constructor for finer control.\n\n```go\n// 1. Backbone with default US (existing call — unchanged)\npool, _ := proxypool.NewWebshare(apiKey)\n\n// 2. Multi-country via API filter + username injection\npool, _ := proxypool.NewWebshareWithConfig(apiKey, proxypool.WebshareConfig{\n    Countries: []string{\"US\", \"GB\", \"DE\"},\n})\n\n// 3. Rotating endpoint — no API key needed, Webshare rotates IPs internally\npool, _ := proxypool.NewWebshareRotating(os.Getenv(\"WEBSHARE_USER\"), os.Getenv(\"WEBSHARE_PASS\"), \"US\")\n```\n\nThe pool round-robins across all entries. With multiple countries, each base proxy is duplicated per country so rotations naturally spread across geographies.\n\n### As http.RoundTripper\n\n```go\nclient, _ := stealth.NewClient()\nresp, err := client.StdClient().Get(\"https://example.com\")\n```\n\n### Middleware\n\n```go\nclient, _ := stealth.NewClient()\nclient.Use(stealth.LoggingMiddleware)\nclient.Use(stealth.RetryMiddleware(stealth.DefaultRetryConfig))\nclient.Use(stealth.RateLimitMiddleware(ratelimit.NewLimiter(ratelimit.DefaultConfig)))\n```\n\n### Rate Limiting\n\n```go\nlimiter := ratelimit.NewDomainLimiter(ratelimit.DomainConfig{\n    Rules: map[string]ratelimit.Config{\n        \"api.example.com\": {RequestsPerWindow: 10, WindowDuration: time.Minute},\n        \"*.example.com\":   {RequestsPerWindow: 30, WindowDuration: time.Minute},\n    },\n})\nlimiter.Wait(ctx, \"https://api.example.com/v1/users\")\n```\n\n### Generic Pool\n\n```go\npool := pool.New(accounts, pool.Config{\n    AlertHook: func(topic string, payload any) { log.Println(topic, payload) },\n})\nacc, err := pool.Next(func(a *Account) bool { return a.IsReady() })\n```\n\n## Packages\n\n| Package | Purpose |\n|---------|---------|\n| `stealth` | BrowserClient, middleware, profiles, retry, backoff |\n| `pool` | Generic `Pool[T Identity]` with health tracking |\n| `proxypool` | ProxyPool interface + Static, Webshare, HealthyProxyPool |\n| `ratelimit` | Per-key sliding window + per-domain limiter |\n| `session` | Stateful browsing with persistence |\n| `internal/fingerprint` | Reference types + oracle comparison for the fingerprint measurement |\n| `cmd/fingerprint-capture` | Captures a real Chrome's fingerprint as an oracle reference |\n\n## Fingerprint oracle\n\n`make fingerprint` runs the TLS/HTTP2 fingerprint oracle, which checks that each\nChrome profile in `BuiltinProfiles` actually emits the fingerprint a real Chrome\nof the same major version emits. It is **not** part of `make preflight` (it hits\nthe network and needs reference files); run it explicitly.\n\nA **failure** means a go-stealth Chrome profile's emitted fingerprint differs\nfrom a real Chrome's — a true result (the profile is stale or wrong), not a test\ndefect. Fix the profile in a separate reviewed change; do not weaken the\ncomparison to make it green.\n\nThe oracle compares each metric against a service that is spec-faithful for that\nmetric: **JA4** (and `ja4_o` / `ja3n_hash`) against **browserleaks**\n(FoxIO-faithful — peet.ws strips the padding extension 0x0015 from JA4), and\n**JA3**, **peetprint**, **HTTP/2 Akamai**, **header order**, and **sec-ch-ua**\nagainst **peet** (spec-faithful JA3; the only service with peetprint and\nsent-frames). Each reference records per-metric provenance in `sources`, and the\noracle FAILs if a reference and a measurement for the same metric come from\ndifferent services — a cross-service comparison reports a tooling artefact as a\nfingerprint defect.\n\nReferences live in `testdata/reference_chrome_\u003cmajor\u003e.json`, captured by:\n\n```bash\ngo run ./cmd/fingerprint-capture -major 146   # amd64 host; no arm64 Chrome-for-Testing build\n```\n\nSee `testdata/README.md` for the headless caveat and the per-metric provenance\ncontract.\n\n## Used By\n\n- [go-twitter](https://github.com/anatolykoptev/go-twitter) — Twitter/X scraping\n- [go-threads](https://github.com/anatolykoptev/go-threads) — Threads.net scraping\n- [go-search](https://github.com/anatolykoptev/go-search) — Web search MCP server\n- [go-hully](https://github.com/anatolykoptev/go-hully) — Crypto intelligence\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanatolykoptev%2Fgo-stealth","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanatolykoptev%2Fgo-stealth","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanatolykoptev%2Fgo-stealth/lists"}