{"id":50391069,"url":"https://github.com/tgdrive/varc","last_synced_at":"2026-05-30T18:01:39.080Z","repository":{"id":334059418,"uuid":"1139838724","full_name":"tgdrive/varc","owner":"tgdrive","description":"Range-caching HTTP reverse proxy. Caches byte ranges on disk, parallel chunked downloads, Caddy module","archived":false,"fork":false,"pushed_at":"2026-05-17T13:55:51.000Z","size":303,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-17T15:52:04.478Z","etag":null,"topics":["cache-proxy","caching","caddy","go","http-proxy","range-request","streaming"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tgdrive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-22T13:38:45.000Z","updated_at":"2026-05-17T13:58:53.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tgdrive/varc","commit_stats":null,"previous_names":["tgdrive/vfscache-proxy","tgdrive/rclone-vfs"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/tgdrive/varc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgdrive%2Fvarc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgdrive%2Fvarc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgdrive%2Fvarc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgdrive%2Fvarc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tgdrive","download_url":"https://codeload.github.com/tgdrive/varc/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tgdrive%2Fvarc/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33703065,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache-proxy","caching","caddy","go","http-proxy","range-request","streaming"],"created_at":"2026-05-30T18:01:37.699Z","updated_at":"2026-05-30T18:01:39.075Z","avatar_url":"https://github.com/tgdrive.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# varc\n\n`varc` is a Caddy v2 HTTP handler module for serving remote HTTP objects through the `varc` sparse read-through VFS cache.\n\nThe important behavior is **cache-first range serving**:\n\n1. Build the varc key from the request.\n2. Try `varc.Open(..., nil, varc.WithCacheOnly())`.\n3. Parse the client `Range` header using cached metadata.\n4. If the byte range is already present, serve from local disk immediately.\n5. Only on miss, probe/fetch the upstream with HTTP `Range` requests.\n\nThat means already-cached video/media ranges do **not** create an upstream client/request.\n\n## Layout\n\n```text\nvarc/\n  go.mod\n  handler.go       # Caddy HTTP handler module\n  caddyfile.go     # Caddyfile parser\n  source.go        # HTTP Range -\u003e io.ReaderAt adapter\n  range.go         # Range / validator handling\n  logger.go\n  varc/varc.go     # embedded varc package from your implementation\n  examples/Caddyfile\n```\n\n## Build with xcaddy\n\nFrom this directory:\n\n```bash\nxcaddy build \\\n  --with github.com/tgdrive/varc=.\n```\n\nThen run:\n\n```bash\n./caddy run --config examples/Caddyfile\n```\n\n## Caddyfile\n\n```caddyfile\n:8080 {\n    route /media/* {\n        varc https://origin.example.com {\n            cache_dir /var/cache/caddy/varc\n            key {host}:{uri}\n\n            # Request URL behavior. With append_uri on, /media/a.mp4 is\n            # fetched from https://origin.example.com/media/a.mp4.\n            append_uri on\n            ignore_query off\n\n            # Cache tuning.\n            block_size 1MiB\n            chunk_size 128MiB\n            chunk_streams 4\n            max_inflight_bytes 512MiB\n            read_ahead 16MiB\n            max_size 500GiB\n            max_age 168h\n            poll_interval 1m\n            shard_level 2\n\n            # Origin HTTP tuning.\n            timeout 60s\n            probe_timeout 15s\n            dial_timeout 10s\n            response_header_timeout 30s\n            max_idle_conns 256\n\n            # Optional origin auth / tenancy.\n            # header Authorization \"Bearer {$ORIGIN_TOKEN}\"\n            # forward_header Authorization\n\n            # Debug/admin.\n            debug_headers on\n            admin_path /_varc\n        }\n    }\n}\n```\n\n## Cache-only mode\n\nUse this when Caddy must only serve data that is already present in varc:\n\n```caddyfile\n:8080 {\n    route /media/* {\n        varc https://origin.example.com {\n            cache_dir /var/cache/caddy/varc\n            cache_only on\n            pass_thru off\n        }\n    }\n}\n```\n\nOn cache hit, no upstream HTTP request is made. On miss, the handler returns a miss/error unless `pass_thru on` is enabled.\n\n## JSON config\n\n```json\n{\n  \"apps\": {\n    \"http\": {\n      \"servers\": {\n        \"srv0\": {\n          \"listen\": [\":8080\"],\n          \"routes\": [{\n            \"match\": [{\"path\": [\"/media/*\"]}],\n            \"handle\": [{\n              \"handler\": \"varc\",\n              \"upstream\": \"https://origin.example.com\",\n              \"cache_dir\": \"/var/cache/caddy/varc\",\n              \"key\": \"{http.request.host}:{http.request.uri}\",\n              \"chunk_size\": 134217728,\n              \"block_size\": 1048576,\n              \"chunk_streams\": 4,\n              \"read_ahead\": 16777216,\n              \"cache_max_size\": 536870912000,\n              \"debug_headers\": true\n            }]\n          }]\n        }\n      }\n    }\n  }\n}\n```\n\n## Headers\n\nThe module always sets:\n\n- `Accept-Ranges: bytes`\n- `Content-Length`\n- `Content-Range` for `206 Partial Content`\n- `ETag` when the origin provides one\n- `Last-Modified` when the origin provides one\n\nWith `debug_headers on`, it also sets:\n\n- `X-Varc-Cache: HIT|MISS`\n- `X-Varc-Key`\n- `X-Varc-Source`\n\n## Admin endpoint\n\nIf `admin_path /_varc` is configured:\n\n```bash\ncurl http://localhost:8080/_varc\ncurl -X POST 'http://localhost:8080/_varc?action=prune'\n```\n\nProtect this route with Caddy matchers/auth if exposed outside localhost.\n\n## Important production notes\n\n- The handler assumes the upstream object is byte-addressable with HTTP `Range` support.\n- It forces `Accept-Encoding: identity` so cached byte offsets map to real upstream bytes.\n- Cached hits intentionally skip upstream validation. This is what avoids client initialization/network calls for already cached ranges. Use stable keys, ETags/fingerprints, TTL pruning, or versioned URLs to avoid serving stale content.\n- Multi-range responses are rejected with `416`. Most media players issue single ranges.\n- If your origin requires authorization, either add a static `header` or list request headers with `forward_header`. Include auth scope in `key` if different users may see different bytes for the same URL.\n\nFor Caddy adapter validation:\n\n```bash\n./caddy adapt --config examples/Caddyfile --pretty\n```\n\n## Go library\n\nThe `varc` package is a standalone sparse read-through range cache for Go programs — media servers, object-storage gateways, FUSE/VFS layers, or any workload that reads byte ranges from a slower `io.ReaderAt` source.\n\nMinimal usage:\n\n```go\ncache, _ := varc.New(ctx, varc.Options{CacheDir: \"./.cache\"})\ndefer cache.Close()\n\n// Anything that implements io.ReaderAt can be a source.\nr, _ := cache.Open(ctx, key, size, src, varc.WithFingerprint(etag))\ndefer r.Close()\n\nbuf := make([]byte, 64\u003c\u003c10)\nn, _ := r.ReadAt(buf, offset)\n```\n\nSee [API_USAGE.md](API_USAGE.md) for the full Go library reference: cache keys, fingerprints, readers, warming, metrics, pruning, HTTP range source adapter, error handling, and production tuning.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftgdrive%2Fvarc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftgdrive%2Fvarc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftgdrive%2Fvarc/lists"}