{"id":30449999,"url":"https://github.com/deflect-ca/banjax","last_synced_at":"2025-08-23T13:20:16.354Z","repository":{"id":37097768,"uuid":"399514852","full_name":"deflect-ca/banjax","owner":"deflect-ca","description":"Rate limitor written in Go for HTTP requests according to a configurable set of regex patterns","archived":false,"fork":false,"pushed_at":"2025-06-05T19:58:52.000Z","size":1002,"stargazers_count":2,"open_issues_count":2,"forks_count":1,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-07-17T14:09:42.826Z","etag":null,"topics":["baskerville","docker","golang","kafka","nginx"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deflect-ca.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"supporting-containers/nginx/Dockerfile","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-08-24T15:23:16.000Z","updated_at":"2025-06-23T14:32:22.000Z","dependencies_parsed_at":"2023-09-27T13:16:36.777Z","dependency_job_id":"3d9c5c0a-1fee-42bf-a1db-06a1aad5d919","html_url":"https://github.com/deflect-ca/banjax","commit_stats":null,"previous_names":[],"tags_count":44,"template":false,"template_full_name":null,"purl":"pkg:github/deflect-ca/banjax","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fbanjax","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fbanjax/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fbanjax/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fbanjax/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deflect-ca","download_url":"https://codeload.github.com/deflect-ca/banjax/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fbanjax/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271749047,"owners_count":24814114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-23T02:00:09.327Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["baskerville","docker","golang","kafka","nginx"],"created_at":"2025-08-23T13:20:14.001Z","updated_at":"2025-08-23T13:20:16.341Z","avatar_url":"https://github.com/deflect-ca.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Banjax\n\n## Table of Contents\n1. [What is Banjax](#what-is-banjax)\n    1. [Backstory](#backstory)\n    2. [Banjax-Go](#banjax-go)\n    3. [Technology](#technology)\n2. [How it Looks in a Deflect Environment](#how-it-looks-in-a-deflect-environment)\n3. [Decision-Making Process](#decision-making-process)\n4. [Password-protected paths](#password-protected-paths)\n5. [Installation](#installation)\n    1. [Talking to banjax-go from an existing Nginx (or other) setup](#talking-to-banjax-go-from-an-existing-nginx-or-other-setup)\n    2. [Deploying Nginx + banjax-go in front of an existing web server](#deploying-nginx--banjax-go-in-front-of-an-existing-web-server)\n    3. [Running the new Nginx + banjax-go on the same host as your existing server](#running-the-new-nginx--banjax-go-on-the-same-host-as-your-existing-server)\n    4. [Running the new Nginx + banjax-go on a host other than your existing server](#running-the-new-nginx--banjax-go-on-a-host-other-than-your-existing-server)\n6. [Talking to Baskerville over a Kafka connection](#talking-to-baskerville-over-a-kafka-connection)\n7. [Sample Configuration](#sample-configuration)\n\n# What is Banjax\n\n## Backstory\n\nOur edge architecture at Deflect for many years was Apache Trafficserver and a custom plugin called Banjax. This plugin did some basic rate-limiting on incoming requests according to a configurable set of regex patterns. If a rate-limit was exceeded, we could block further requests from that IP address, or we could serve a challenge-response page (JS-based proof-of-work, or password-based authentication).\n\nThis old plugin-based system was hooking Trafficserver\u0026#39;s internal request-processing state machine events. This was quite hard to understand and extend, and the resulting behavior wasn\u0026#39;t apparent or changeable in a config file.\n\n## Banjax-Go\n\nBanjax-go is a rewrite of this which performs the same functionality, but as a separate process rather than a plugin. We\u0026#39;re also using it with Nginx now instead of Trafficserver, but the idea should work with any reverse proxy that supports X-Accel-Redirect. Instead of doing the regex matching on incoming requests in the Nginx/Trafficserver process, we are tailing access logs similarly to fail2ban.\n\nLeveraging existing higher-level HTTP middleware concepts makes the whole thing easier to understand and modify. We can leverage Nginx\u0026#39;s powerful block-based configuration format. Caching the auth request responses, adding timeout and failure-handling behavior, or treating static files specially can all be done in the Nginx config instead of with code.\n\n## Technology\n\n- [Go](https://golang.org/)\n- [Docker](https://www.docker.com/) (each service/component is run in its own docker container)\n- [Nginx](https://www.nginx.com/)\n- [Kafka](https://hub.docker.com/r/wurstmeister/kafka/)\n- [Baskerville](https://github.com/deflect-ca/baskerville) (optional, can be used together with Banjax to detect anomalous traffic)\n\n# How it Looks in a Deflect Environment\n\nThe diagram below highlights the request life-cycle inside a fully-deployed Deflect DDoS infrastructure. Note the role of Banjax during a request life-cycle, and how it can also interact with other systems, such as Baskerville.\n\n![](docs/request.png)\n\n# Decision-Making Process\n\nBanjax-go currently has four internal Decision types:\n\n- Allow\n- returns X-Accel-Redirect: @access\\_granted to Nginx.\n- NginxBlock\n- returns X-Accel-Redirect: @access\\_denied to Nginx.\n- IptablesBlock\n- returns @access\\_denied _and also_ blocks that IP with iptables.\n- Challenge\n- returns a JS-based SHA-inverting proof-of-work page. if the request contains a cookie with a solved challenge, banjax-go returns @access\\_granted. if an IP exceeds a rate limit of failed challenges, they get blocked.\n\nThe Decision lists are populated by:\n\n- The config file. This is useful for allowlisting or blocklisting known good or bad IPs.\n- The regex-matching rate-limiting log tailer. Rules specify the regex to look for, the number of hits and time interval that determine the rate limit, and the Decision to take for future requests from that IP if the rate limit is exceeded.\n- A Kafka topic. This is how Baskerville sends its ML-informed commands to Banjax-go.\n- A rate limit on the number of failed challenges an IP submits. Bots will generally fail a bunch of challenges, and we want to block them after a while rather than serve them an unlimited number of challenge pages.\n\nDecisions added at run-time (from the log tailer, Kafka, or the failed challenge rate limit) expire after some configurable amount of time.\n\n\u003cimg width=\"1384\" alt=\"Banjax-flowchart\" src=\"https://github.com/user-attachments/assets/ca28fc7f-bd28-49a1-8bbb-473cfdf759dd\" /\u003e\n\n# Password-protected paths\n\nFrom the perspective of the JS and cookie cryptographic implementation, these work very similarly to the SHA-inverting proof-of-work challenge. But the use-cases are different: the PoW challenge is intended to filter out DDoS traffic, and so it makes sense for the Nginx configuration to fail open in case Banjax-go is unreachable. Password-protected paths should fail closed.\n\n# Development\n\nCreate a empty `.env` file for docker-compose. `touch .env`\n\n```\nexport ENABLE_AIR=1  # enable hot-reloading for dev\ndocker-compose up --build\n```\n\n# Installation\n\nYou should be able to get a quick demo of banjax-go + Nginx + a test origin server\nrunning with `docker-compose up`. (See also `docker-compose.prod.yml`)\n\nYou're probably running an existing Nginx (or similar) server. We describe two options\nfor adding banjax-go to your setup. The first involves changing your existing Nginx\nconfiguration to talk to banjax-go. The second involves adding a new Nginx server in\nfront of your existing one.\n\n## Talking to banjax-go from an existing Nginx (or other) setup\n\nYou'll want to read the sample `nginx.conf` under `supporting_containers/nginx` and understand\nhow it works.\n\nAn incoming request matches a location block like this. The `proxy_pass` directive\nsends the request to banjax-go.\n```\nlocation / {\n    proxy_pass http://127.0.0.1:8081/auth_request?;\n}\n```\n\nbanjax-go responds with one of:\n* A challenge page containing JS that will set cookies for authenticating subsequent requests.\n  This response gets sent back to the client.\n* A response with `X-Accel-Redirect` set to `@access_granted` or `@access_denied`. This header\n  tells Nginx not to respond to the client yet, but to perform an internal redirect to one\n  of the location blocks called `@access_granted` or `@access_denied`.\n```\nlocation @access_denied {\n    return 403 \"access denied\";\n}\n\nlocation @access_granted {\n    proxy_pass http://test-origin:8080;\n}\n```\n\nWe're only using this with Nginx, but other proxy servers seem to have a similar mechanism.\n\n## Deploying Nginx + banjax-go in front of an existing web server\n\nThe easiest way to get started here would be to edit the sample `nginx.conf` under\n`supporting_containers/nginx` to point to your origin server (change all the\n`proxy_pass http://test-origin:8080;` lines).\n\nThen use the provided `docker-compose.yml` to start Dockerized instances of Nginx and\nbanjax-go: `docker-compose up --build nginx banjax-next`.\n\nNow you can make a request through this new Nginx with:\n```\ncurl --header \"Host: example.com\" http://127.0.0.1:80\n```\nor\n```\ncurl --resolve example.com:80:127.0.0.1 http://127.0.0.1:80\n```\n\nFor non-HTTPS requests like shown here, these commands will do the same thing\n(namely, ensure an HTTP Host: header gets sent to Nginx so it knows which server\nblock to use). For HTTPS requests, the second form of the command will also ensure\nthe hostname gets sent in the TLS SNI field so that Nginx knows which server key to\nuse for the TLS connection. For a single-site Nginx configuration with\n`server_name _;`, you might not have to worry about any of this.\n\n## Running the new Nginx + banjax-go on the same host as your existing server\n\nYou'll want to change your existing server to listen on, for example, 127.0.0.1:8080, and\nconfigure the new Nginx to `proxy_pass` to that local address.\n\nBinding on 127.0.0.1 will let the new Nginx connect to it over the loopback interface,\nbut will keep others from being to connect to it over the internet.\n\nThen configure the new Nginx to listen on 0.0.0.0:80 (and 443).\n\n## Running the new Nginx + banjax-go on a host other than your existing server\n\nYou'll need to point your DNS records to the new host's IP address.\n\nAnd you'll configure the new Nginx to point to your existing server's IP address and port.\n\nAnd you'll want to use a firewall on the existing host to that only the new Nginx is allowed\nto connect to it.\n\n# Talking to Baskerville over a Kafka connection\n\nbanjax-go can optionally send and receive messages over a Kafka bus. We use this to\ncommunicate with our Baskerville ML anomaly detection system.\n\nMessages sent on `kafka_command_topic` (from Baskerville to banjax-go):\n* `{\"name\": \"challenge_ip\", \"value\": \"1.2.3.4\"}`\n  * Tells banjax-go to challenge this IP the next time it's seen.\n\nMessages sent on `kafka_report_topic` (from banjax-go to Baskerville):\n* `{\"name\": \"ip_passed_challenge\", \"value_ip\": \"1.2.3.4\", \"value_site\": \"example.com\"}`\n  * Tells Baskerville an IP passed a challenge.\n* `{\"name\": \"ip_failed_challenge\", [same as above]...}`\n  * Tells Baskerville an IP failed a challenge.\n* `{\"name\": \"ip_banned\", [same as above]...}`\n  * Tells Baskerville an IP failed enough challenges to get banned.\n\nThe banning threshold is configured with:\n```yaml\ntoo_many_failed_challenges_interval_seconds: 10\ntoo_many_failed_challenges_threshold: 3\n```\n\nThe Kafka connection is configured with:\n```yaml\nkafka_brokers:\n  - \"localhost:9092\"\nkafka_security_protocol: 'ssl'\nkafka_ssl_ca: \"/etc/banjax/caroot.pem\"\nkafka_ssl_cert: \"/etc/banjax/certificate.pem\"\nkafka_ssl_key: \"/etc/banjax/key.pem\"\nkafka_ssl_key_password: password\nkafka_report_topic: 'banjax_report_topic'\nkafka_command_topic: 'banjax_command_topic'\n```\n\n# Sample Configuration\n\nNote: See `banjax-config.yaml` in the repo for full example\n\n```yaml\nconfig_version: 2020-12-15_12:35:38\nglobal_decision_lists:         # static allow/challenge/block decisions (global)\n  allow:\n  - 20.20.20.20\n  block:\n  - 30.40.50.60\n  challenge:\n  - 8.8.8.8\nper_site_decision_lists:       # static allow/challenge/block decisions (per-site)\n  example.com:\n    allow:\n    - 20.20.20.20\n    block:\n    - 30.40.50.60\n    challenge:\n    - 8.8.8.8\niptables_ban_seconds: 10       # how long an iptables ban lasts\niptables_unbanner_seconds: 5   # how often the unbanning task runs\nkafka_brokers:\n- localhost:9092\npassword_hashes:               # for password_protected_paths\n  example.com: \u003cbase64 string\u003e \npassword_protected_paths:      # for password_protected_paths\n  example.com:\n  - wp-admin\nper_site_regexes_with_rates: # fail2ban-like challenging/blocking (per-site)\n  example.com:\n  - decision: block\n    hits_per_interval: 10\n    interval: 120\n    name: UNNAMED RULE\n    regex: 'GET \\/search\\/.*'\nregexes_with_rates:            # fail2ban-like challenging/blocking (global)\n- decision: block\n  hits_per_interval: 0\n  interval: 1\n  regex: .*blockme.*\n  rule: instant block\n- decision: challenge\n  hits_per_interval: 0\n  interval: 1\n  regex: .*challengeme.*\n  rule: instant challenge\nserver_log_file: /var/log/banjax/banjax-format.log  # nginx log file with specific format\n```\n\n---\n\n\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by/4.0/\"\u003e\n\u003cimg alt=\"Creative Commons Licence\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by/4.0/80x15.png\" /\u003e\u003c/a\u003e\u003cbr /\u003e\nThis work is copyright (c) 2020, eQualit.ie inc., and is licensed under a \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by/4.0/\"\u003eCreative Commons Attribution 4.0 International License\u003c/a\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeflect-ca%2Fbanjax","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeflect-ca%2Fbanjax","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeflect-ca%2Fbanjax/lists"}