{"id":27615641,"url":"https://github.com/kiyoshi-work/resilient-proxy-out","last_synced_at":"2026-05-11T05:35:00.296Z","repository":{"id":288418638,"uuid":"965329151","full_name":"kiyoshi-work/resilient-proxy-out","owner":"kiyoshi-work","description":"A resilient API proxy built with OpenResty/NGINX+Lua featuring circuit breakers, request caching, retry mechanisms, and comprehensive monitoring dashboards. Designed to protect applications from third-party API failures while providing detailed usage statistics.","archived":false,"fork":false,"pushed_at":"2025-04-20T11:15:37.000Z","size":59,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-23T03:56:59.616Z","etag":null,"topics":["lua","nginx","openresty","redis"],"latest_commit_sha":null,"homepage":"","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kiyoshi-work.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-12T23:10:27.000Z","updated_at":"2025-04-20T11:15:40.000Z","dependencies_parsed_at":"2025-04-18T01:53:20.985Z","dependency_job_id":"626a43ac-b73e-43b7-8fb2-4b154f601cdf","html_url":"https://github.com/kiyoshi-work/resilient-proxy-out","commit_stats":null,"previous_names":["kiyoshi-work/resilient-proxy-out"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiyoshi-work%2Fresilient-proxy-out","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiyoshi-work%2Fresilient-proxy-out/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiyoshi-work%2Fresilient-proxy-out/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiyoshi-work%2Fresilient-proxy-out/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kiyoshi-work","download_url":"https://codeload.github.com/kiyoshi-work/resilient-proxy-out/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250366715,"owners_count":21418769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lua","nginx","openresty","redis"],"created_at":"2025-04-23T03:57:03.291Z","updated_at":"2026-05-11T05:35:00.290Z","avatar_url":"https://github.com/kiyoshi-work.png","language":"Lua","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Resilient Proxy Out\n\nA resilient proxy built with OpenResty (a powerful web platform that extends NGINX with Lua scripting capabilities), designed to provide caching, rate limiting, and proxying capabilities for third party APIs.\n\n## Features\n\n- **Circuit Breaker Pattern**: Automatically detects failing services and prevents cascading failures, the circuit breaker can be configured per API with the following options:\n    - `failure_threshold`: Number of failures before circuit is tripped (default: 5)\n    - `reset_timeout`: Time in seconds before circuit is reset (default: 30)\n    - `request_timeout`: Request timeout in milliseconds (default: 10000)\n    - `success_threshold`: Number of successful requests before circuit is reset (default: 2)\n- **Request Caching**: Reduces load on backend services by caching responses in Redis, the cache can be configured per API with the following options:\n    - `enable_cache`: Enable caching (default: false)\n    - `cache_ttl`: Cache time-to-live in seconds (default: 60)\n    - `cache_header_strategy`: Strategy to use for caching headers (default: \"none\")\n    - `cache_headers`: Headers to use for caching (default: \"none\")\n- **Proxy Support**: Optional proxy configuration for handling rate-limited APIs, the proxy can be configured per API with the following options:\n    - `use_proxy`: Enable proxy (default: false)\n    - `proxy_strategy`: proxy selection strategy (default: `round_robin`). One of:\n        - `round_robin` — distribute every request across the full proxy list. Maximum IP utilization, lowest keepalive pool hit rate.\n        - `sticky` — hash a key (configurable via `proxy_sticky_key`) to a fixed proxy. Same key always lands on the same proxy → keepalive pool stays warm per key. On `429`, the retry falls back to `round_robin` to escape the limited proxy.\n        - `subset` — round-robin within the first N proxies (configurable via `proxy_subset_size`). Traffic concentrates onto fewer proxies → higher pool hit rate. On `429`, the active window slides to the next N proxies.\n        - `on_rate_limit` — direct connection by default; only switch to a proxy after a `429` is observed.\n        - `never` — never use a proxy.\n    - `proxy_sticky_key` (only when `proxy_strategy = \"sticky\"`): key source for the hash. One of `client_ip` (default), `api_name`, or `header:\u003cname\u003e`. Avoid low-cardinality keys like `api_name` alone — they collapse all traffic onto one proxy and defeat per-IP rate limiting.\n    - `proxy_subset_size` (only when `proxy_strategy = \"subset\"`): number of proxies in the active window (default: 3).\n- **Retry Mechanism**: Automatically retries failed requests with exponential backoff , the retry mechanism can be configured per API with the following options:\n\n    - `max_attempts`: Maximum number of retry attempts (default: 3)\n    - `initial_delay`: Initial delay in seconds before first retry (default: 1)\n    - `max_delay`: Maximum delay in seconds between retries (default: 10)\n    - `backoff_factor`: Exponential backoff multiplier (default: 2)\n    - `retry_on_status`: HTTP status codes that trigger a retry (default: 500, 502, 503, 504, 429)\n    - `retry_on_errors`: Connection errors that trigger a retry (default: timeout, connection refused, etc.)\n- **Detailed API Statistics**: Comprehensive statistics tracking for all API calls with a visual dashboard:\n    - **Path-Level Statistics**: Track and analyze API usage at both the API and individual path levels\n    - **Response Time Metrics**: Monitor min, median, 95th percentile, and max response times\n    - **Status Code Distribution**: Visualize the distribution of HTTP status codes\n    - **Error Tracking**: Log and display recent error messages for troubleshooting\n    - **Time-Based Analysis**: View statistics for all time, daily, or hourly periods\n    - **Auto-Refresh**: Configure automatic dashboard updates at customizable intervals\n\n- **Dashboards \u0026 Monitoring**: The proxy includes several built-in dashboards for monitoring and troubleshooting:\n  - **Statistics Dashboard**: `/stats-dashboard` - path-level statistics, response time metrics, status code distribution, error tracking, time-based analysis (all/daily/hourly), and auto-refresh capabilities\n  - **Circuit Breaker Dashboard**: `/cb-dashboard` - Real-time circuit status (closed/open/half-open), failure counts, configuration details, and reset timers\n\n\n## Getting Started\n\n### Prerequisites\n\n- Docker and Docker Compose\n- Git\n\n### Installation\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/kiyoshitaro/resilient-proxy-out.git\n   cd resilient-proxy-out\n   ```\n\n2. Configure your environment:\n   ```bash\n   cp .env.sample .env\n   ```\n   \n3. Edit the `.env` file to set your configuration:\n   ```\n   PROXY_URL=https://your-proxy-url.com\n   REDIS_HOST=redis\n   REDIS_PORT=6378\n   ```\n\n4. Start the services:\n   ```bash\n   docker compose up --build -d --remove-orphans\n   ```\n\n5. Verify the installation:\n   ```bash\n   curl http://localhost:8087/health\n   ```\n\n### Project Structure\n```bash\ndocker/\n├── openresty/\n│   ├── conf.d/proxy.conf    # Nginx proxy config\n│   ├── html/*.html    # Circuit Breaker Dashboard\n│   ├── lua/\n│   │   ├── api_config.lua        # API configuration\n│   │   ├── api_proxy.lua        # Main proxy logic\n│   │   └── utils.lua        # Utility functions\n│   ├── Dockerfile\n│   └── nginx.conf\n├── tests/\n│   └── test_*.sh    # Test script for circuit breaker\n├── README.md\n└── .env.sample\n```\n\n## Usage\n### Making API Requests\nSend requests to the gateway on port 8087:\n\n```bash\ncurl 'http://localhost:8087/api/hyperliquid/info' \\\n  -H 'Content-Type: application/json' \\\n  --data-raw '{\"type\":\"frontendOpenOrders\",\"user\":\"0x5887de8d37c9c2550a4d0b86127c43b2e1904545\"'\n```\n### Health Checks\n```bash\ncurl http://localhost:8087/health\n```\n\n## Notes\n\n### Tối ưu latency khi đi qua static proxy IPs (keepalive connection pool)\n\n#### Bài toán\n\nHệ thống dùng 1 list static proxy IPs (rotate qua biến `PROXY_URLS`) để bypass rate limit per-IP của các third-party API. Mỗi request đi:\n\n```\nclient → openresty → proxy IP → upstream API\n```\n\nVì có thêm 1 hop network qua proxy, latency tăng đáng kể. Phần lớn overhead đến từ:\n\n1. **TCP handshake** với proxy mỗi request (1 RTT).\n2. **TLS handshake end-to-end** với upstream (qua `CONNECT` tunnel của HTTP proxy) — tối thiểu 1-2 RTT cho TLS 1.3, 2 RTT cho TLS 1.2.\n3. **Proxy authentication** (Basic auth) lặp lại.\n\nMục tiêu: reuse TCP connection (và TLS tunnel) tới proxy giữa các request → bỏ handshake → giảm latency.\n\n#### Implement cũ — không work\n\n```lua\n-- File: docker/openresty/lua/api_proxy.lua (đoạn cũ)\nlocal httpc = http.new()\n...\nif use_proxy_for_this_request then\n    httpc:set_keepalive(1000, 5)  -- ❌ SAI\n    ...\n    local res, err = httpc:request_uri(full_url, current_request_options)\nend\n```\n\nCó 2 vấn đề chính:\n\n**1. `set_keepalive` gọi sai chỗ và sai semantics.**\n\nTrong `lua-resty-http`, `set_keepalive(timeout, pool_size)` **không phải config setter**. Nó là method \"trả connection hiện tại về pool để reuse\" — phải gọi **sau** khi đã đọc xong response body. Trong code cũ, nó được gọi **trước** `request_uri` lúc `httpc` vừa `http.new()` còn chưa connect → fail silently (không có connection nào để pool). Intent (`keepalive timeout 1000ms, pool 5`) bị hiểu nhầm hoàn toàn — không có dòng config nào thực sự áp dụng.\n\n**2. Round-robin strategy giết pool hit rate.**\n\n`request_uri` thực ra **có** auto-pool nội bộ (gọi `set_keepalive` sau khi đọc body xong). Pool key bao gồm `(host, port, proxy_opts, ssl)`. Khi rotate qua N proxy:\n\n- M req/s tổng → mỗi proxy nhận M/N req/s.\n- Idle time giữa 2 request cùng pool key (cùng proxy) lớn → connection expire trước khi reuse → pool hit ~ 0.\n- Mỗi request gần như đều phải TCP + TLS handshake lại.\n\nVới HTTPS qua HTTP proxy còn tệ hơn: mỗi tunnel `CONNECT` cần TLS handshake end-to-end với upstream, không reuse được = mất 2-3 RTT thừa mỗi request.\n\n#### Fix\n\n```lua\n-- File: docker/openresty/lua/api_proxy.lua\nlocal current_request_options = table_clone(request_options)\n\n-- Bật connection pooling ở level request_uri.\n-- request_uri tự động trả connection về pool sau khi đọc body,\n-- key bởi (host, port, proxy, ssl).\ncurrent_request_options.keepalive_timeout = 60000  -- giữ 60s trong pool\ncurrent_request_options.keepalive_pool = 50        -- max 50 idle conn / pool key\n```\n\nĐồng thời xóa `httpc:set_keepalive(1000, 5)` đặt sai chỗ.\n\nGlobal pool đã có sẵn trong `nginx.conf`:\n\n```nginx\nlua_socket_keepalive_timeout 60s;\nlua_socket_pool_size 100;\n```\n\n#### Tại sao chỉ fix code chưa đủ — cần đổi proxy strategy\n\nSau patch, pool work đúng về kỹ thuật. Nhưng nếu giữ nguyên `round_robin` thuần, pool hit rate vẫn thấp do request bị xé đều ra N proxy. Đã thêm 2 strategy mới: `sticky` và `subset`.\n\n##### `sticky` — hash-based selection\n\nHash 1 stable key (client IP, header, api name) → mapping deterministic vào 1 proxy cố định. Cùng key → cùng proxy → cùng pool key → keepalive reuse rất cao.\n\nConfig:\n\n```lua\nproxy_strategy = \"sticky\",\nproxy_sticky_key = \"client_ip\",   -- \"client_ip\" | \"api_name\" | \"header:\u003cname\u003e\"\n```\n\nImplementation: `ngx.crc32_short(key) % #proxies + 1`.\n\nTrade-off: rate limit per-IP dễ đập hơn. 1 client burst quá hạn → 429 vì luôn đi 1 proxy. Mitigation: trên `429`, retry path tự động fallback sang `round_robin` để thoát proxy bị limit.\n\nPhù hợp khi: traffic per-client thấp hơn nhiều so với rate limit per-IP của upstream.\n\n##### `subset` — active window\n\nRound-robin trong subset N proxy đầu (default 3) thay vì full list. Traffic dồn vào ít proxy hơn → pool hit rate tăng. Trên `429`, offset của window dịch lên 1 → swap proxy bị limit ra ngoài subset.\n\nConfig:\n\n```lua\nproxy_strategy = \"subset\",\nproxy_subset_size = 3,\n```\n\nPhù hợp khi: traffic vừa phải, cần balance giữa pool warmth và headroom với rate limit.\n\n##### Bảng so sánh nhanh\n\n| Strategy | Pool hit | IP utilization | 429 risk | Phù hợp |\n|----------|----------|----------------|----------|---------|\n| `round_robin` | Thấp | Tốt nhất | Thấp nhất | Traffic rất cao, cần rate limit budget tối đa |\n| `sticky` | Cao nhất | Tệ (1 client = 1 IP) | Cao cho hot client | Per-client traffic nhỏ |\n| `subset` | Trung bình-cao | Trung bình | Trung bình, có swap | Default an toàn cho hầu hết case |\n| `on_rate_limit` | N/A khi direct | N/A | N/A | Chỉ proxy khi cần thiết |\n| `never` | N/A | N/A | N/A | Local/internal API |\n\n##### Case Hyperliquid (`api.hyperliquid.xyz`)\n\nRate limit upstream: **600 req/min per IP = 10 rps per IP**.\n\nMath:\n\n- `round_robin` với N proxy: aggregate ~ N * 10 rps. IP utilization tốt nhưng pool hit rate ~ 0 (request xé đều).\n- `sticky` by `client_ip`: 1 client max = 10 rps. Trên 10 rps/client → 429 liên tục dù aggregate còn dư.\n- `subset(3)`: 3 proxy active = 30 rps aggregate. Pool warm cho 3 host. 429 ở 1 proxy → window slide, proxy đó out, proxy mới in.\n\nConfig đã chọn:\n\n```lua\nhyperliquid = {\n    target_url = \"https://api.hyperliquid.xyz\",\n    use_proxy = true,\n    proxy_strategy = \"subset\",\n    proxy_subset_size = 3,\n    ...\n}\n```\n\nTuning:\n- Peak aggregate \u003e 30 rps → tăng `proxy_subset_size` (mỗi +1 = +10 rps headroom, giảm pool hit rate).\n- Peak aggregate \u003c\u003c 10 rps → giảm xuống 1 hoặc dùng `sticky`/`on_rate_limit` để pool hit cực đại.\n- Per-client traffic dominate (1 client \u003e 10 rps) → cần kết hợp app-level throttle, không strategy nào fix được giới hạn 10 rps/IP nếu client spam vào cùng key.\n\n#### Verify\n\n- Log `res.connection_reused` (nếu `lua-resty-http` version có expose) hoặc bật debug log của resty.http.\n- `tcpdump -i any 'tcp[tcpflags] \u0026 tcp-syn != 0'` filter theo IP proxy: nếu pool work, số SYN packet phải giảm mạnh so với số request.\n- So sánh `p50`/`p95` latency trước/sau patch qua `benchmark.sh`.\n\n#### Tunables\n\n| Param | Giá trị | Ý nghĩa |\n|-------|---------|---------|\n| `keepalive_timeout` | 60000 ms | Connection idle trong pool 60s trước khi đóng |\n| `keepalive_pool` | 50 | Max idle conn per `(host, port, proxy)` key |\n| `lua_socket_pool_size` | 100 | Global cap per worker (nginx.conf) |\n| `lua_socket_keepalive_timeout` | 60s | Global default (nginx.conf) |\n\nTăng `keepalive_pool` nếu concurrent request per proxy cao. Tăng `keepalive_timeout` nếu traffic burst với khoảng cách lớn — nhưng để ý proxy/firewall có thể đóng idle connection ở phía họ.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a PR.\n\n## License\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiyoshi-work%2Fresilient-proxy-out","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkiyoshi-work%2Fresilient-proxy-out","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiyoshi-work%2Fresilient-proxy-out/lists"}