{"id":20359679,"url":"https://github.com/n0vad3v/autorl","last_synced_at":"2025-04-12T03:32:38.493Z","repository":{"id":58071791,"uuid":"487446166","full_name":"n0vad3v/AutoRL","owner":"n0vad3v","description":"Automatically block traffic on Cloudflare's side based on Nginx Log parsing.","archived":false,"fork":false,"pushed_at":"2024-03-13T07:56:49.000Z","size":101,"stargazers_count":60,"open_issues_count":0,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-25T23:08:50.846Z","etag":null,"topics":["cloudflare","cloudflare-api","cloudflare-waf"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/n0vad3v.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-05-01T04:38:57.000Z","updated_at":"2024-09-19T04:08:24.000Z","dependencies_parsed_at":"2022-09-17T04:21:34.219Z","dependency_job_id":null,"html_url":"https://github.com/n0vad3v/AutoRL","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/n0vad3v%2FAutoRL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/n0vad3v%2FAutoRL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/n0vad3v%2FAutoRL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/n0vad3v%2FAutoRL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/n0vad3v","download_url":"https://codeload.github.com/n0vad3v/AutoRL/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248512747,"owners_count":21116674,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudflare","cloudflare-api","cloudflare-waf"],"created_at":"2024-11-14T23:35:51.388Z","updated_at":"2025-04-12T03:32:38.474Z","avatar_url":"https://github.com/n0vad3v.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AutoRL\n\nThis is a PoC of automatically block traffic on Cloudflare's side based on Nginx Log parsing.\n\nIt will evaluate Nginx `access.log` and find potential CC pattern, and block them on Cloudflare's side and send a message(alert) to Telegram Group.\n\n## Topology\n\nWith Cloudflare Argo Tunnel, we can set security group to allow inbound traffic for SSH only, this can guarantee the Host's IP will not be exposed to the Internet (ref: [Use Cloudflare Argo Tunnel (cloudflared) to accelerate and protect your website.](https://nova.moe/accelerate-and-secure-with-cloudflared-en/)), however, attackers can still CC your website by sending enormous requests cocurrently, AutoRL is here trying to mitigate this problem.\n\n![](./AutoRL.png)\n\n## Prerequisites\n\nSince this is only a PoC, the following condition must be met to use AutoRL.\n\n* Python 3 installed on Host\n* Nginx used for Reverse proxy and all the logs are logged into one `access.log` file.\n* Nginx has the following log format (in `/etc/nginx/nginx.conf`)\n\n    ```\n    log_format  main  '$remote_addr $time_iso8601 \"$request\" $server_name '\n                      '$status $body_bytes_sent \"$http_referer\" '\n                      '\"$http_user_agent\" \"$http_x_forwarded_for\"';\n    ```\n    On this condition, the raw log should look like this:\n    ```\n    108.162.245.152 2022-05-05T10:14:19+08:00 \"GET /grafana/api/live/ws HTTP/1.1\" xxxx.yyyy.tld 400 12 \"-\" \"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)\" \"145.xx.xxx.xxx\"\n    ```\n    Where, `108.162.245.152` is Cloudflare's IP,`xxxx.yyyy.tld` is the requested domain , `2022-05-02T10:44:16+08:00` stands for request datetime and `\"145.xx.xx.xxx\"` is the real visitor IP.\n\n\n## Usage\n\n1. Download the `autorl.py` to your host\n2. Edit the following variable in the `autorl.py`\n\n    * CF_EMAIL (Your Cloudflare login email)\n    * CF_AUTH_KEY (Your Cloudflare Global API Key)\n    * ACCESS_LOG_PATH (Default is `/var/log/nginx/access.log`)\n    * INTERVAL_MIN (Default is 1, then this script will evaluate for 1min's traffic)\n    * RATE_PER_MINUTE (How many requests are allowed for single IP, e,g, when this is set to 600 and `INTERVAL_MIN` is 1, then one IP can send at most 600 requests, after that, this IP will be blocked.)\n    * TG_CHAT_ID (Your Telegram Chat Group ID, optional)\n    * TG_BOT_TOKEN (You should invite a bot to your group, and fillin the bot token here, optional)\n    * IP_WHITE_LIST (If you'd like to whitelist some IP, fillin here)\n3. Create a crontab for this script, example:\n    ```\n    * * * * * for i in {1..6}; do /usr/bin/python3 /path/to/autorl.py \u0026 sleep 10; done\n    ```\n\n## Demo\n\nOn Telegram side:\n\n![](./demo.png)\n\nOn Cloudflare side:\n\n![](./demo-cf.png)\n\n## Notes\n\n* Blocked IP address will never gets unblocked.\n* If logrotate is not setup correctly, then parsing the whole `access.log` might consume a lot of system resources.\n* The attack pattern/sample is not stored so we have no idea how the attack is conducted.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fn0vad3v%2Fautorl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fn0vad3v%2Fautorl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fn0vad3v%2Fautorl/lists"}