{"id":44818420,"url":"https://github.com/tternquist/beyond-ads-dns","last_synced_at":"2026-04-01T17:33:47.502Z","repository":{"id":337050679,"uuid":"1152170973","full_name":"tternquist/beyond-ads-dns","owner":"tternquist","description":"Ad-blocking DNS resolver that uses public blocklists (e.g. Hagezi) and Redis caching to reduce upstream traffic. Performance optimized.","archived":false,"fork":false,"pushed_at":"2026-03-28T11:28:51.000Z","size":9631,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-28T15:17:32.191Z","etag":null,"topics":["adblocker","blocker","dns","dns-over-https","dns-over-tls","dns-server"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tternquist.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-07T13:24:40.000Z","updated_at":"2026-03-28T11:16:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"b4bd7755-e3bd-48b5-9b7c-4a552c6b6738","html_url":"https://github.com/tternquist/beyond-ads-dns","commit_stats":null,"previous_names":["tternquist/beyond-ads-dns"],"tags_count":124,"template":false,"template_full_name":null,"purl":"pkg:github/tternquist/beyond-ads-dns","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tternquist%2Fbeyond-ads-dns","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tternquist%2Fbeyond-ads-dns/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tternquist%2Fbeyond-ads-dns/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tternquist%2Fbeyond-ads-dns/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tternquist","download_url":"https://codeload.github.com/tternquist/beyond-ads-dns/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tternquist%2Fbeyond-ads-dns/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290537,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adblocker","blocker","dns","dns-over-https","dns-over-tls","dns-server"],"created_at":"2026-02-16T19:12:17.911Z","updated_at":"2026-04-01T17:33:47.488Z","avatar_url":"https://github.com/tternquist.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo.svg\" alt=\"BEYOND-ADS-DNS Logo\" width=\"400\"\u003e\n\u003c/p\u003e\n\n# beyond-ads-dns\n\nAd-blocking DNS resolver that uses public blocklists (e.g. Hagezi)\nand Redis caching to reduce upstream traffic. Performance optimized.\n\n**AI-Human Collaboration:** This repository is an AI-assisted project. While AI was used to generate the boilerplate and initial code implementation, the product roadmap, system design, and overall architecture were human-directed. We believe in leveraging AI for efficiency while maintaining human oversight for quality and intent.\n\n## Running the application\n\n**Recommended: Docker Compose.** The easiest way to run beyond-ads-dns is with one of the Docker Compose examples. They include Redis, ClickHouse, and the Metrics UI—no manual setup required.\n\n| Example | Summary | Link |\n|---------|---------|------|\n| **Basic** | Minimal deployment using the published GHCR image (no build). Redis, ClickHouse, Metrics UI. | [`examples/basic-docker-compose/`](examples/basic-docker-compose/) |\n| **Appliance + Watchtower** | For devices outside your control. Uses `appliance` tag; Watchtower auto-updates when you promote a new version. | [`examples/appliance-with-watchtower/`](examples/appliance-with-watchtower/) |\n| **Let's Encrypt** | Automatic HTTPS for the Metrics UI via Let's Encrypt. | [`examples/letsencrypt-docker-compose/`](examples/letsencrypt-docker-compose/) |\n| **Grafana** | Adds Prometheus and Grafana for monitoring, dashboards, and query analytics. | [`examples/grafana-integration/`](examples/grafana-integration/) |\n| **Max Performance** | Tuned for high throughput (2GB Redis, 100K L0 cache, higher batch sizes). | [`examples/max-performance-docker-compose/`](examples/max-performance-docker-compose/) |\n| **Raspberry Pi** | MicroSD-friendly: ClickHouse in memory only (tmpfs), tmpfs for Redis and logs. | [`examples/raspberry-pi-docker-compose/`](examples/raspberry-pi-docker-compose/) |\n| **Unbound** | Unbound as recursive upstream—no third-party DNS, full DNSSEC validation. | [`examples/unbound-docker-compose/`](examples/unbound-docker-compose/) |\n| **Redis Sentinel** | Learning: Redis HA with master-replica and automatic failover. | [`examples/redis-sentinel-docker-compose/`](examples/redis-sentinel-docker-compose/) |\n| **Redis Cluster** | Learning: Redis Cluster with sharded data and failover. | [`examples/redis-cluster-docker-compose/`](examples/redis-cluster-docker-compose/) |\n| **Source build** | Build image from source (not standard—use only for custom code, dev, or restricted environments). | [`examples/source-build-docker-compose/`](examples/source-build-docker-compose/) |\n| **Kubernetes (Helm)** | Deploy on Kubernetes with optional NodePort or hostNetwork for DNS. Redis/ClickHouse external or via dependencies. | [`helm/beyond-ads-dns/`](helm/beyond-ads-dns/) |\n\n**Quick start (Basic example):**\n\n```bash\ncd examples/basic-docker-compose\ndocker compose up -d\n```\n\n- **DNS**: `localhost:53` (UDP/TCP)\n- **Metrics UI**: http://localhost\n- **Control API**: http://localhost:8081\n\n### Alternative: Non-Docker installation\n\nIf you cannot use Docker, you can run the resolver and its dependencies manually.\n\n**Prerequisites:**\n\n- **Go 1.24+** (to build the binary)\n- **Redis** (required for DNS cache)\n- **ClickHouse** (optional; disable with `query_store.enabled: false` in config)\n\n**1. Build the binary:**\n\n```bash\ngo build -o beyond-ads-dns ./cmd/beyond-ads-dns\n```\n\n**2. Create and edit config:**\n\n```bash\ncp config/config.example.yaml config/config.yaml\n```\n\nEdit `config/config.yaml` and set addresses for your local services:\n\n- `cache.redis.address`: Redis address (e.g. `localhost:6379`)\n- `cache.redis.password`: Redis password if your Redis uses `requirepass` (see [Redis password setup](docs/redis-password-setup.md))\n- `query_store.address`: ClickHouse HTTP address (e.g. `http://localhost:8123`) if using query store\n- `query_store.username` / `query_store.password`: ClickHouse credentials (prefer `QUERY_STORE_PASSWORD` env; see [ClickHouse password setup](docs/clickhouse-password-setup.md))\n\nIf you disable the query store, set `query_store.enabled: false` and omit ClickHouse setup.\n\n**3. Set up Redis and ClickHouse (if needed):**\n\n- **Redis**: Install Redis (e.g. `apt install redis-server` on Debian/Ubuntu) and ensure it is running.\n- **ClickHouse**: Install ClickHouse and create the schema from `db/clickhouse/init.sql`. Default credentials in the example config are `beyondads`/`beyondads`.\n\n**4. Run the resolver:**\n\n```bash\n./beyond-ads-dns -config config/config.yaml\n```\n\nThe Metrics UI (React app + Node.js API) is bundled in the Docker image. For non-Docker setups, run it separately—see the [Metrics UI](#metrics-ui) section.\n\n### Kubernetes (Helm)\n\nTo run on Kubernetes, use the included Helm chart. You must provide a Redis URL; ClickHouse is optional.\n\n```bash\nhelm install beyond-ads-dns ./helm/beyond-ads-dns --set redis.url=redis://your-redis:6379\n```\n\nDNS can be exposed via **NodePort** (e.g. `\u003cnode-ip\u003e:3053`) or **hostNetwork** (port 53 on the node). See [helm/beyond-ads-dns/README.md](helm/beyond-ads-dns/README.md) for configuration, persistence, and optional ClickHouse init.\n\n## Architecture (data structures + algorithms)\n\n### Blocklist compilation and matching\n\n- **Sources**: blocklists are fetched on a schedule and parsed line‑by‑line.\n- **Normalization**: each line is trimmed, comments removed, and common\n  list formats are supported (hosts file lines, `||domain^` rules,\n  AdBlock-style with `$important`, `$script`, `|https://domain^`, etc).\n  Domains are lower‑cased, trailing dots removed, and `*.` stripped.\n- **Storage**: entries are stored in an in‑memory hash set\n  `map[string]struct{}` for O(1) lookups.\n- **Overrides**:\n  - `allowlist` entries are stored in a separate set and always win.\n  - `denylist` entries are always blocked, even if not in blocklists.\n- **Matching algorithm**: the query name is normalized and checked for\n  suffix matches by progressively stripping left‑most labels\n  (`ads.example.com` → `example.com` → `com`). This allows a single\n  list entry to match subdomains efficiently.\n\n### Cache layout and refresh\n\n- **Cache key**: `dns:\u003cqname\u003e:\u003cqtype\u003e:\u003cqclass\u003e`\n- **Value**: a Redis hash containing:\n  - `msg`: wire‑encoded DNS response\n  - `soft_expiry`: UNIX epoch for the soft TTL\n- **Expiry index**: a sorted set `dnsmeta:expiry:index` keyed by\n  `soft_expiry` to enable sweep scans.\n- **Metadata**: hit counters and refresh locks use the `dnsmeta:` prefix\n  and may expire; cache entries do not.\n- **Redis eviction policy**: By default, Redis uses `allkeys-lru` (Least\n  Recently Used) to evict keys when maxmemory is reached. This ensures\n  that less-frequently accessed DNS entries are removed first when memory\n  pressure occurs. Cache entries persist without Redis TTLs, so eviction\n  is the only way they are removed from Redis (besides explicit deletes).\n  Redis is configured via `config/redis.conf` (see below for details).\n- **Refresh algorithms**:\n  - **Refresh‑ahead**: on cache hit, refresh if soft TTL is below\n    `min_ttl` or `hot_ttl` (for hot keys).\n  - **Sweeper**: periodically scans the expiry index and refreshes keys\n    close to expiry and with at least `sweep_min_hits` within\n    `sweep_hit_window`.\n- **Stale serving**: expired entries can be served for `stale_ttl`\n  while refresh runs in the background.\n\n### Query store and metrics\n\n- **ClickHouse storage**: each query is inserted as a row with timestamp,\n  client, qname, outcome, and latency. This powers query dashboards.\n- **Response time measurement**: The `duration_ms` metric measures the\n  **complete end-to-end response time** from when the query is received\n  until the response is written back to the client. This includes:\n  - Blocklist checking (if enabled)\n  - Cache lookup time (Redis query)\n  - Upstream DNS query time (if cache miss)\n  - Cache write time (for new entries)\n  - Network time to send the response to the client\n  \n  This is *not* just the upstream DNS response time—it captures the full\n  request processing latency, giving you a complete picture of client\n  experience.\n- **Metrics API**: the Node.js API exposes Redis stats, query summaries,\n  and refresh sweep stats to the UI.\n\n### Control plane\n\n- **Control server**: `/blocklists/reload` applies config changes by\n  reloading blocklists without restarting the DNS service.\n\n## Usage and configuration\n\nThe resolver loads a default config from `config/default.yaml` and then\napplies any overrides found in `config/config.yaml` (gitignored). You can\noverride the user config path with `-config` or `CONFIG_PATH`, and the\ndefault path with `DEFAULT_CONFIG_PATH`.\n\n**Redis address via environment:** `REDIS_ADDRESS` or `REDIS_URL` overrides\n`cache.redis.address`. Use `REDIS_ADDRESS` for host:port (e.g. `redis:6379`)\nor `REDIS_URL` for a full URL (e.g. `redis://redis-node-1:6379`). Useful in\nDocker when the Redis hostname differs from the default.\n\nCreate a user override config to customize blocklists and upstreams:\n\n```\ncp config/config.example.yaml config/config.yaml\n```\n\n### Config overview\n\n```\nserver:\n  listen: [\"0.0.0.0:53\"]\n  protocols: [\"udp\", \"tcp\"]\n  read_timeout: \"5s\"\n  write_timeout: \"5s\"\n\n# Upstreams: plain DNS (host:port), DoT (tls://host:853), DoQ (quic://host:853), or DoH (https://host/dns-query)\nupstreams:\n  - name: cloudflare\n    address: \"1.1.1.1:53\"\n\nblocklists:\n  refresh_interval: \"6h\"\n  sources:\n    - name: hagezi-pro\n      url: \"https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/pro.txt\"\n  allowlist: []\n  denylist: []\n\ncache:\n  redis:\n    address: \"redis:6379\"\n    db: 0\n    password: \"\"\n  min_ttl: \"300s\"\n  max_ttl: \"1h\"\n  negative_ttl: \"5m\"\n  # client_ttl_cap: \"5m\"   # Two-tier TTL: max TTL in client responses when serving from cache. Omit = disabled. Default 5m.\n  refresh:\n    enabled: true\n    hit_window: \"1m\"\n    hot_threshold: 20\n    min_ttl: \"30s\"\n    hot_ttl: \"2m\"\n    serve_stale: true\n    stale_ttl: \"1h\"\n    lock_ttl: \"10s\"\n    max_inflight: 100\n    sweep_interval: \"15s\"\n    sweep_window: \"1m\"\n    max_batch_size: 2000\n    sweep_min_hits: 1\n    sweep_hit_window: \"48h\"\n\nresponse:\n  blocked: \"nxdomain\"   # or IP (e.g. resolver IP) for block page; set to server IP to serve block page for blocked domains\n  blocked_ttl: \"1h\"\n\nrequest_log:\n  enabled: false\n  directory: \"logs\"\n  filename_prefix: \"dns-requests\"\n  format: \"text\"  # or \"json\" for structured logs with query_id\n\n# Client identification: map IPs to names for per-device analytics (Configure → Clients)\n# client_identification:\n#   enabled: true\n#   clients:\n#     - ip: \"192.168.1.10\"\n#       name: \"Kids Tablet\"\n#       group_id: \"kids\"\n# client_groups:\n#   - id: \"kids\"\n#     name: \"Kids\"\n#     description: \"Children's devices\"\n\nquery_store:\n  enabled: true\n  address: \"http://clickhouse:8123\"\n  database: \"beyond_ads\"\n  table: \"dns_queries\"\n  username: \"beyondads\"\n  password: \"beyondads\"\n  flush_to_store_interval: \"5s\"   # How often the app sends buffered events to ClickHouse\n  flush_to_disk_interval: \"5s\"   # How often ClickHouse flushes async inserts to disk\n  batch_size: 2000\n  sample_rate: 1.0  # Fraction to record (0.0-1.0). Use \u003c1.0 to reduce load at scale.\n\ncontrol:\n  enabled: true\n  listen: \"0.0.0.0:8081\"\n  token: \"\"\n\n# Multi-instance sync (optional). Configure via Metrics UI Sync tab or manually:\n# sync:\n#   enabled: true\n#   role: primary   # or replica\n#   # Primary: tokens for replicas (created via UI)\n#   tokens: []\n#   # Replica only:\n#   primary_url: \"http://primary-host:8081\"\n#   sync_token: \"token-from-primary\"\n#   sync_interval: \"60s\"\n```\n\nRequest logging is disabled by default. Set\n`request_log.enabled: true` to enable daily rotation.\nUse `request_log.format: \"json\"` for structured JSON logs with `query_id`, `qname`, `outcome`, and `duration_ms`.\nQuery store supports `query_store.sample_rate` (0.0–1.0) to record a fraction of queries, reducing load at scale.\nSet `query_store.anonymize_client_ip: \"hash\"` or `\"truncate\"` for GDPR/privacy-compliant retention.\n\nCache refresh-ahead is enabled by default. The resolver will\npreemptively refresh hot entries when they are close to expiring. Tune\n`cache.refresh.*` to adjust how aggressive the refresh should be.\nThe sweeper periodically scans for keys nearing expiration and refreshes\nthem, even if they are not actively requested.\nStale serving keeps expired entries available for `cache.refresh.stale_ttl`\nwhile background refreshes keep them up to date.\nCache entries are stored without Redis TTLs; soft expiry is tracked\ninternally so keys persist until Redis evicts them based on the\nconfigured eviction policy (default: `allkeys-lru`).\nMetadata keys (hit counters, locks, sweep index) use the `dnsmeta:`\nprefix and may have TTLs; cache entries keep the `dns:` prefix and do\nnot expire.\n\n#### Redis configuration\n\nRedis is configured via `config/redis.conf`, which is mounted into the\nRedis container. The default configuration sets:\n\n- **`maxmemory 512mb`**: Memory limit before eviction\n- **`maxmemory-policy allkeys-lru`**: Eviction policy\n- **Persistence**: RDB snapshot at most every 5 minutes (`save 300 1`); AOF disabled (DNS cache repopulates quickly on restart)\n\nAvailable eviction policies:\n\n- **`allkeys-lru`** (default): Evict least recently used keys from all keys\n- **`allkeys-lfu`**: Evict least frequently used keys from all keys\n- **`allkeys-random`**: Evict random keys from all keys\n- **`volatile-lru`**: Evict least recently used keys with TTL set\n- **`volatile-lfu`**: Evict least frequently used keys with TTL set\n- **`volatile-random`**: Evict random keys with TTL set\n- **`volatile-ttl`**: Evict keys with TTL set, shortest TTL first\n- **`noeviction`**: Return errors when memory limit is reached\n\nSince DNS cache entries do not have Redis TTLs, `volatile-*` policies will\nonly evict metadata keys (hit counters, locks). For typical DNS caching,\n**`allkeys-lru` or `allkeys-lfu` are recommended** to ensure cache entries\ncan be evicted under memory pressure.\n\nTo customize Redis settings, edit `config/redis.conf` before starting the\ncontainers.\n\nFor production HA, use Redis Sentinel or Cluster:\n\n```yaml\ncache:\n  redis:\n    mode: sentinel\n    master_name: mymaster\n    sentinel_addrs: [\"sentinel1:26379\", \"sentinel2:26379\"]\n    password: \"\"\n    lru_size: 10000\n```\n\nOr Redis Cluster: `mode: cluster` with `cluster_addrs: [\"node1:6379\", \"node2:6379\", ...]`.\n\n**Docker users** can pass Redis config via environment variables (no config file needed):\n\n- **Standalone**: `REDIS_URL` or `REDIS_ADDRESS`\n- **Sentinel**: `REDIS_MODE=sentinel`, `REDIS_MASTER_NAME=mymaster`, `REDIS_SENTINEL_ADDRS=sentinel1:26379,sentinel2:26379` (or `REDIS_ADDRESS` as comma-separated sentinels)\n- **Cluster**: `REDIS_MODE=cluster`, `REDIS_CLUSTER_ADDRS=node1:6379,node2:6379,node3:6379` (or `REDIS_ADDRESS` as comma-separated nodes)\n\n### Cache refresh details\n\nThe cache keeps two notions of expiry:\n\n- **Soft expiry**: the original DNS TTL (after clamping). This decides\n  whether a response is \"fresh\" or \"stale\".\n- **Redis eviction**: keys do not have Redis TTLs. They remain until\n  Redis evicts them based on its configured policy/memory limits.\n\n#### Refresh paths\n\nThere are two refresh mechanisms that can run together:\n\n1. **Request‑driven refresh (refresh‑ahead)**  \n   When a cached entry is served and its soft TTL is low, the resolver\n   refreshes it in the background. The threshold is based on recent\n   request frequency:\n   - If the entry has **query rate ≥ `hot_threshold_rate`** (queries per minute)\n     within `hit_window`, it is treated as \"hot\" and refreshed once its\n     TTL is low. When `client_ttl_cap` is set (default 5m), the default\n     adapts: ~3 clients = hot (2/min for 5m cap); single client stays warm. Without client cap, 20/min.\n     soft TTL is below `hot_ttl`.\n   - Otherwise, it refreshes once soft TTL is below `min_ttl`.\n\n2. **Periodic sweeper**  \n   The sweeper runs every `sweep_interval`, scanning the internal\n   soft‑expiry index for keys expiring within `sweep_window`. It schedules\n   refreshes for keys that are close to expiry **and** that have seen at\n   least `sweep_min_hits` within `sweep_hit_window`. Hits are recorded on\n   cache serves and on successful cache writes after upstream responses.\n   \n   **What happens to entries that don't meet the threshold?**  \n   Entries with fewer than `sweep_min_hits` are **skipped by the sweeper**\n   and are not proactively refreshed. They remain in cache and will:\n   - Continue to be served if still fresh (before soft expiry)\n   - Be served as stale if `serve_stale` is enabled (within `stale_ttl`\n     after soft expiry)\n   - Become unservable after `stale_ttl` expires (hard cache miss)\n   - Persist in Redis until evicted by Redis's memory policy, since cache\n     entries do not have Redis TTLs\n   - Still be eligible for request‑driven refresh if accessed by a client\n\nBoth refresh paths are protected by a **distributed lock** (per key) and\na **local inflight limit**, so a single hot key won’t trigger stampedes.\n\n#### Stale serving\n\nIf `serve_stale` is enabled, the resolver will serve expired entries for\nup to `stale_ttl` **after soft expiry**, while a refresh is scheduled in\nthe background. This avoids a hard cache miss for clients when the entry\nhas just gone stale.\n\n#### Configuration reference\n\n```\ncache:\n  refresh:\n    enabled: true          # Master switch for refresh-ahead + sweeper\n    hit_window: \"1m\"       # Window for counting request frequency\n    hot_threshold: 20      # Absolute fallback when hot_threshold_rate is 0\n    # hot_threshold_rate: auto when client_ttl_cap set (~2/min for 5m cap; single client stays warm); else 20/min\n    min_ttl: \"1h\"          # Refresh threshold for non-hot entries\n    hot_ttl: \"2m\"          # Refresh threshold for hot entries\n    serve_stale: true      # Serve expired entries within stale_ttl\n    stale_ttl: \"1h\"        # Max time to serve stale entries\n    expired_entry_ttl: \"30s\"  # TTL in DNS response when serving expired entries\n    lock_ttl: \"10s\"        # Per-key refresh lock in Redis\n    max_inflight: 100       # Max concurrent refreshes per instance\n    sweep_interval: \"15s\"  # How often the sweeper runs\n    sweep_window: \"1m\"     # How far ahead the sweeper scans\n    max_batch_size: 2000    # Max keys processed per sweep\n    sweep_min_hits: 1      # Min hits in sweep_hit_window to refresh\n    sweep_hit_window: \"48h\" # Time window for sweep_min_hits\n```\n\n#### Tuning guidance\n\n- **More aggressive refresh**: increase `hot_ttl`/`min_ttl`, shorten\n  `sweep_interval`, or increase `sweep_window`.\n- **Less upstream load**: decrease `hot_ttl`/`min_ttl` and increase\n  `hit_window` or `hot_threshold`.\n- **Avoid stampedes**: keep `lock_ttl` \u003e= expected upstream latency and\n  set `max_inflight` to a reasonable limit for your instance.\n\nQuery storage uses ClickHouse and is enabled by default. Set\n`query_store.enabled: false` to disable it.\nThe ClickHouse schema lives in `db/clickhouse/init.sql`. It uses partition-level TTL to reduce disk writes; existing installations should run the migration in `db/clickhouse/PARTITION_TTL_MIGRATION.md`.\nThe default Docker Compose credentials are `beyondads`/`beyondads`.\n\nQuery store flush intervals:\n- **`flush_to_store_interval`** (default `5s`): How often the app sends buffered query events to ClickHouse. Also triggers when `batch_size` is reached.\n- **`flush_to_disk_interval`** (default `5s`): How often ClickHouse flushes its async insert buffer to disk (`async_insert_busy_timeout_ms`). Controls when data is durably persisted.\n\nThe Docker Compose examples disable ClickHouse's internal system logs (query_log, part_log, etc.) to reduce disk writes. If you see persistent `SystemLogFlush` or `MergeMutate` writes in `iotop`, see [docs/clickhouse-disk-writes.md](docs/clickhouse-disk-writes.md).\n\nThe control server is used by the UI to apply blocklist changes. If you\nset `control.token`, the UI must send the same token via\n`DNS_CONTROL_TOKEN` in the metrics API.\n\n#### DoH/DoT server (encrypted client connections)\n\nTo accept DNS-over-HTTPS and DNS-over-TLS from clients, enable the DoH/DoT server with TLS certificates:\n\n```yaml\ndoh_dot_server:\n  enabled: true\n  cert_file: \"/path/to/fullchain.pem\"\n  key_file: \"/path/to/privkey.pem\"\n  dot_listen: \"0.0.0.0:853\"   # DoT (DNS over TLS)\n  doh_listen: \"0.0.0.0:8443\"  # DoH (DNS over HTTPS); use 443 if not sharing with web UI\n  doh_path: \"/dns-query\"\n```\n\nClients can then use `tls://your-host:853` for DoT or `https://your-host:8443/dns-query` for DoH.\n\n#### Block page\n\nWhen `response.blocked` is set to your resolver's IP (e.g. the host running the Metrics UI), blocked domains resolve to that IP. The Metrics UI serves a simple HTML block page when a browser requests a blocked domain. Configure `response.blocked` to your server's IP and ensure DNS points clients to your resolver.\n\n## Performance\n\nThe resolver uses a multi-tier caching architecture for maximum performance:\n\n- **L0 Cache**: In-memory LRU cache (~10-50μs latency)\n- **L1 Cache**: Redis distributed cache (~0.5-2ms latency)\n- **Bloom Filter**: Fast negative lookups for blocklists\n- **Refresh-Ahead**: Proactive cache refresh to avoid expiry\n- **Optimized Connection Pools**: Redis pool with 50 connections\n\nExpected performance with default configuration:\n- **Hot queries**: \u003c0.1ms latency, 500K-1M QPS per instance\n- **Cached queries**: 0.5-2ms latency, 50K-100K QPS per instance\n- **Cache hit rate**: 95-99% in production\n\nSee [`docs/performance.md`](docs/performance.md) for detailed performance documentation and tuning guide. For metered or low-bandwidth deployments, see [Network Bandwidth Configuration](docs/network-bandwidth-configuration.md) for how to balance configuration options and reduce network usage.\n\n## Testing\n\nAutomated tests include Go unit tests, web server (Node.js) tests, and web client (Vitest) tests. All run in CI on every push and pull request.\n\n```bash\ngo test ./...                           # Go tests\nnpm test --prefix web/server            # Web server API tests\nnpm test --prefix web/client            # Web client unit tests\n```\n\nSee [`docs/testing.md`](docs/testing.md) for full documentation: test locations, coverage, and how to add new tests. For an overview of architecture and code practices (for AI agents and developers), see [`docs/code-and-architecture-standards.md`](docs/code-and-architecture-standards.md).\n\n\n## Docker\n\nThe Docker image combines the DNS resolver and metrics API in a single\ncontainer. Redis and ClickHouse run as separate services. **No config files\nare required**—the image includes sensible defaults (Hagezi blocklist,\nCloudflare upstreams).\n\nTo build the image from source (e.g. for custom deployments), use the [source-build Docker Compose example](examples/source-build-docker-compose/) or run `docker build -t beyond-ads-dns .` from the repo root. **This is not the standard approach**—for most users, use one of the other [Docker Compose examples](#running-the-application) above; they use the published image from GitHub Container Registry and require no build.\n\n**Image tags**: `stable` (manually promoted, recommended for production), `appliance` (for Watchtower-monitored deployments—promote validated releases via [Promote to Appliance Tag](.github/workflows/appliance-tag.yml)), `latest` (auto-updates with each release), `v1.2.3` (pinned version), `edge` (bleeding-edge from main). Use the [Promote to Stable Tag](.github/workflows/stable-tag.yml) workflow to selectively promote a validated release to `stable`.\n\nTo customize blocklists or upstreams, use the Metrics UI—changes save to\n`./config/config.yaml` on the host. Default config is in the image; no default.yaml required.\nSet `HOSTNAME` in `.env` or configure hostname in **Settings → UI** to customize the hostname shown in the UI. The Settings hostname overrides `HOSTNAME` when both are set.\n\n## Metrics UI\n\nThe metrics UI is a React app backed by a Node.js API, bundled in the\nsame Docker image as the DNS resolver. It surfaces Redis cache\nstatistics, recent query rows, blocklist management, instance sync\nconfiguration, and the active configuration (when the control server is enabled). The query table\nsupports filtering, pagination, sorting, and CSV export. **Configure → Local Records** provides a dedicated, Route53-inspired interface for managing local DNS records (table view, filtering, record details panel).\n\nRun via one of the [Docker Compose examples](#running-the-application) (recommended; e.g. `examples/basic-docker-compose`). Visit http://localhost for the Metrics UI.\n\nFor non-Docker setups, run the web server and client separately:\n\n```bash\ncd web/server \u0026\u0026 npm install \u0026\u0026 npm run dev\ncd web/client \u0026\u0026 npm install \u0026\u0026 npm run dev\n```\n\n### Instance Sync\n\nThe Metrics UI includes a **Sync** tab to configure multi-instance sync. You can enable sync, choose a role (primary or replica), and manage settings entirely from the UI—no manual config editing required.\n\n**Primary instance** (source of truth):\n\n- Create sync tokens for replicas to authenticate\n- Revoke tokens when replicas are decommissioned\n- Blocklists, upstreams, and local records are managed here and pushed to replicas\n\n**Replica instance**:\n\n- Set the primary URL (e.g. `http://primary-host:8081`)\n- Enter the sync token from the primary\n- Configure sync interval (e.g. `60s`, `5m`)\n- DNS-affecting config (blocklists, upstreams, local records) is read-only; it is synced from the primary\n\nTo get started: open the Sync tab, choose **Primary** or **Replica**, and follow the prompts. Restart the application after saving to apply changes.\n\n### Settings\n\nThe **Settings** tab configures server, cache, query store, control, logging, and request log. Key options:\n\n- **Application Logging** — Format (text/JSON) and level (error/warning/info/debug). Use **JSON** for Grafana/Loki integration.\n- **Control → Error persistence** — Persist errors to disk for the Error Viewer.\n- **Request Log** — Log DNS requests to disk (text or JSON).\n- **Clients \u0026 Groups** — Map client IPs to friendly names and assign to groups for per-device analytics and future per-group blocklists (parental controls). Manage via Configure → Clients. Applies immediately, no restart. See [Clients and Groups](docs/clients-and-groups.md).\n\nMost settings require a restart to take effect.\n\n## Grafana Integration\n\nThe resolver exposes Prometheus metrics at `http://localhost:8081/metrics` when the control server is enabled. To run with Grafana and Prometheus, use the [Grafana integration example](examples/grafana-integration/):\n\n```bash\ncd examples/grafana-integration\ndocker compose up --build\n```\n\nThen open Grafana at http://localhost:3000 (login: `admin` / `admin`). Prometheus, ClickHouse, and Loki datasources are pre-configured. See [`examples/grafana-integration/README.md`](examples/grafana-integration/README.md) for details. For application logs in Grafana, set **Settings → Application Logging → Format** to **JSON** in the Metrics UI and restart.\n\nLocal development:\n\n```\ncd web/server \u0026\u0026 npm install \u0026\u0026 npm run dev\ncd web/client \u0026\u0026 npm install \u0026\u0026 npm run dev\n```\n\n## Security\n\nThe metrics UI supports optional login, sessions, and HTTPS.\n\n### User/Password login\n\nWhen a password is configured, the UI requires login before accessing any data.\n\n**Set password via environment variable:**\n\n```yaml\n# In your docker-compose.yml (e.g. examples/basic-docker-compose/docker-compose.yml)\nenvironment:\n  - UI_PASSWORD=your-secure-password\n```\n\nOr use `ADMIN_PASSWORD` (alias) or `UI_USERNAME` to customize the admin username (default: `admin`).\n\n**Set password via command in container:**\n\n```bash\ndocker exec beyond-ads-dns beyond-ads-dns set-admin-password your-secure-password\n```\n\nOr run interactively (prompts for password):\n\n```bash\ndocker exec -it beyond-ads-dns beyond-ads-dns set-admin-password\n```\n\nThe command writes a bcrypt hash to `/app/config-overrides/.admin-password`. Override the path with `ADMIN_PASSWORD_FILE`.\n\n**Set or change password via UI:**\n\nWhen the password is stored in a file (not set via `UI_PASSWORD` or `ADMIN_PASSWORD` env), you can set or change it from **System Settings** in the web UI:\n\n- **Initial setup:** If no password is configured, go to System Settings and set a password. The UI will reload and require login.\n- **Change password:** When logged in, go to System Settings, enter your current password and the new password, then click \"Change password\".\n\nIf the password is configured via environment variables, it cannot be changed from the UI.\n\n**Initial setup and persistence:**\n\n- By default, new installations have no password. Set one via the UI or CLI before exposing the dashboard on a shared network.\n- When using file-based auth, the password file must be on a **persistent volume**. In Docker Compose examples, `./config:/app/config-overrides` ensures the password persists across container restarts. Without this volume, the password would be lost on restart.\n\n### Sessions\n\nSessions are stored in Redis with a configurable secret. Set `SESSION_SECRET` in production for stable session signing. Cookie is httpOnly, sameSite=lax, and secure when HTTPS is enabled.\n\n### HTTPS\n\n**Option 1: Let's Encrypt (automatic)**\n\nFor automatic HTTPS with free certificates from Let's Encrypt:\n\n```yaml\nenvironment:\n  - LETSENCRYPT_ENABLED=true\n  - LETSENCRYPT_DOMAIN=your-domain.com\n  - LETSENCRYPT_EMAIL=admin@your-domain.com\n```\n\n**HTTP challenge** (default): Port 80 must be publicly reachable. **DNS challenge** (alternative): Set `LETSENCRYPT_CHALLENGE_TYPE=dns` when port 80 is not available—add TXT records manually when prompted. Certificates are stored in a volume and auto-renew on startup when expiring. See [`examples/letsencrypt-docker-compose/`](examples/letsencrypt-docker-compose/) for a full example.\n\n**Option 2: Manual certificates**\n\n```yaml\nenvironment:\n  - HTTPS_ENABLED=true\n  - SSL_CERT_FILE=/path/to/cert.pem\n  - SSL_KEY_FILE=/path/to/key.pem\n  - HTTPS_PORT=443\n```\n\nMount the certificate files into the container. Alternatively, use a reverse proxy (nginx, Traefik) for TLS termination.\n\n## Performance testing\n\nUse the built-in harness to run large query bursts and optionally flush\nRedis between runs:\n\n```\ngo run ./cmd/perf-tester -resolver 127.0.0.1:53 -flush-redis -control-url http://127.0.0.1:8081\n```\n\nSee `tools/perf/README.md` for more options (warmup, TCP, custom name\nlists, etc).\n\n**Note on latency measurements**: The performance tester measures\n**client-side round-trip time** (from sending query to receiving\nresponse, including network latency). This differs from the\nserver-side `duration_ms` stored in ClickHouse, which measures\nend-to-end processing time within the resolver (see \"Query store and\nmetrics\" section above for details).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftternquist%2Fbeyond-ads-dns","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftternquist%2Fbeyond-ads-dns","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftternquist%2Fbeyond-ads-dns/lists"}