{"id":23652770,"url":"https://github.com/fabriziosalmi/patterns","last_synced_at":"2026-06-13T03:09:06.082Z","repository":{"id":269107466,"uuid":"906448944","full_name":"fabriziosalmi/patterns","owner":"fabriziosalmi","description":"Automated OWASP CRS and Bad Bot Detection for Nginx, Apache, Traefik and HaProxy","archived":false,"fork":false,"pushed_at":"2026-06-03T01:16:44.000Z","size":1426,"stargazers_count":307,"open_issues_count":0,"forks_count":7,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-06-03T03:24:15.752Z","etag":null,"topics":["apache","bad-requests","bot-detection","caddy","caddyserver","crs","firewall-configuration","firewall-rules","malicious-url-detection","mod-security","nginx","owasp","waf","web-application-firewall"],"latest_commit_sha":null,"homepage":"https://fabriziosalmi.github.io/patterns/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabriziosalmi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"fabriziosalmi"}},"created_at":"2024-12-21T00:00:15.000Z","updated_at":"2026-05-24T19:12:32.000Z","dependencies_parsed_at":"2026-01-02T11:01:18.396Z","dependency_job_id":null,"html_url":"https://github.com/fabriziosalmi/patterns","commit_stats":null,"previous_names":["fabriziosalmi/patterns"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/fabriziosalmi/patterns","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabriziosalmi%2Fpatterns","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabriziosalmi%2Fpatterns/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabriziosalmi%2Fpatterns/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabriziosalmi%2Fpatterns/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabriziosalmi","download_url":"https://codeload.github.com/fabriziosalmi/patterns/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabriziosalmi%2Fpatterns/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34270447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-13T02:00:06.617Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache","bad-requests","bot-detection","caddy","caddyserver","crs","firewall-configuration","firewall-rules","malicious-url-detection","mod-security","nginx","owasp","waf","web-application-firewall"],"created_at":"2024-12-28T17:01:25.264Z","updated_at":"2026-06-13T03:09:06.062Z","avatar_url":"https://github.com/fabriziosalmi.png","language":"Python","funding_links":["https://github.com/sponsors/fabriziosalmi"],"categories":["Python"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/public/logo.svg\" alt=\"Patterns\" width=\"64\" height=\"64\" /\u003e\n  \u003ch1\u003ePatterns\u003c/h1\u003e\n  \u003cp\u003e\u003cstrong\u003eProduction-grade WAF rules, on autopilot.\u003c/strong\u003e\u003c/p\u003e\n  \u003cp\u003e\n    Automated \u003ca href=\"https://github.com/coreruleset/coreruleset\"\u003eOWASP Core Rule Set\u003c/a\u003e and\n    bad-bot patterns, converted into native configurations for\n    \u003cstrong\u003eNginx\u003c/strong\u003e, \u003cstrong\u003eApache\u003c/strong\u003e, \u003cstrong\u003eTraefik\u003c/strong\u003e, and \u003cstrong\u003eHAProxy\u003c/strong\u003e\n    \u0026mdash; refreshed every day.\n  \u003c/p\u003e\n  \u003cp\u003e\n    \u003ca href=\"https://github.com/fabriziosalmi/patterns/releases/latest\"\u003e\u003cimg alt=\"Latest Release\" src=\"https://img.shields.io/github/v/release/fabriziosalmi/patterns?label=release\u0026color=0071e3\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/fabriziosalmi/patterns/actions/workflows/update_patterns.yml\"\u003e\u003cimg alt=\"Update workflow\" src=\"https://github.com/fabriziosalmi/patterns/actions/workflows/update_patterns.yml/badge.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/fabriziosalmi/patterns/actions/workflows/test_nginx.yml\"\u003e\u003cimg alt=\"Nginx tests\" src=\"https://github.com/fabriziosalmi/patterns/actions/workflows/test_nginx.yml/badge.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"LICENSE\"\u003e\u003cimg alt=\"License\" src=\"https://img.shields.io/badge/license-MIT-1d1d1f\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://fabriziosalmi.github.io/patterns/\"\u003e\u003cimg alt=\"Documentation\" src=\"https://img.shields.io/badge/docs-online-0071e3\"\u003e\u003c/a\u003e\n  \u003c/p\u003e\n  \u003cp\u003e\n    \u003ca href=\"https://fabriziosalmi.github.io/patterns/\"\u003eDocumentation\u003c/a\u003e\n    \u0026middot;\n    \u003ca href=\"https://fabriziosalmi.github.io/patterns/getting-started\"\u003eGet started\u003c/a\u003e\n    \u0026middot;\n    \u003ca href=\"https://github.com/fabriziosalmi/patterns/releases/latest\"\u003eLatest release\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n---\n\n## Why Patterns\n\nThe OWASP Core Rule Set (CRS) is the de-facto open-source rule base behind ModSecurity, but plugging it into anything other than Apache is non-trivial. Patterns automates the whole pipeline:\n\n1. Pull the latest CRS rules straight from upstream.\n2. Convert them into the **native** syntax of each web server \u0026mdash; not a generic shim.\n3. Package the output as ready-to-deploy archives, refreshed every day by GitHub Actions.\n\nYou get equivalent protection across SQL injection, XSS, RCE, LFI, and bad-bot traffic, regardless of which proxy you run.\n\n## Highlights\n\n| | |\n|---|---|\n| **OWASP CRS coverage** | SQLi, XSS, RCE, LFI, RFI, plus generic anomaly and protocol-violation rules. |\n| **Native output** | Nginx `map`/`if`, Apache `SecRule`, Traefik middleware TOML, HAProxy ACL files. |\n| **Bad-bot blocking** | Curated User-Agent lists from public sources, with safe defaults that do **not** block major search engines. |\n| **Daily refresh** | A scheduled GitHub Actions workflow rebuilds every backend and publishes a fresh release. |\n| **Pre-built archives** | Skip the toolchain \u0026mdash; download `nginx_waf.zip`, `apache_waf.zip`, `traefik_waf.zip`, or `haproxy_waf.zip`. |\n| **Composable** | Each backend is a small Python converter on top of one JSON intermediate. Adding a new platform is a few hundred lines. |\n\n\u003e Using **Caddy**? See the dedicated [`caddy-waf`](https://github.com/fabriziosalmi/caddy-waf) project.\n\n## Quick start\n\n### Option 1 \u0026mdash; download a pre-built release\n\n```bash\n# Pick the archive that matches your stack\ncurl -LO https://github.com/fabriziosalmi/patterns/releases/latest/download/nginx_waf.zip\nunzip nginx_waf.zip -d /etc/nginx/waf_patterns\n```\n\nThen follow the [Nginx](https://fabriziosalmi.github.io/patterns/nginx),\n[Apache](https://fabriziosalmi.github.io/patterns/apache),\n[Traefik](https://fabriziosalmi.github.io/patterns/traefik), or\n[HAProxy](https://fabriziosalmi.github.io/patterns/haproxy) integration guide.\n\n### Option 2 \u0026mdash; build from source\n\nRequires **Python 3.11+**, `pip`, and `git`.\n\n```bash\ngit clone https://github.com/fabriziosalmi/patterns.git\ncd patterns\npip install -r requirements.txt\n\npython owasp2json.py            # 1. Fetch the latest OWASP CRS into owasp_rules.json\npython json2nginx.py            # 2. Convert into Nginx WAF config\npython json2apache.py           #    …or Apache (ModSecurity)\npython json2traefik.py          #    …or Traefik middleware\npython json2haproxy.py          #    …or HAProxy ACL files\npython badbots.py               # 3. Generate bad-bot blocklists\n```\n\nGenerated files land in `waf_patterns/\u003cplatform\u003e/`.\n\n## Architecture\n\n```text\n   ┌─────────────────────┐    daily cron     ┌──────────────────────┐\n   │ coreruleset/        │ ───────────────▶  │ owasp2json.py        │\n   │ coreruleset (GH)    │                   │   → owasp_rules.json │\n   └─────────────────────┘                   └──────────┬───────────┘\n                                                        │\n            ┌─────────────────┬──────────────────┬──────┴──────────┐\n            ▼                 ▼                  ▼                 ▼\n      json2nginx.py    json2apache.py    json2traefik.py    json2haproxy.py\n            │                 │                  │                 │\n            ▼                 ▼                  ▼                 ▼\n       nginx_waf.zip    apache_waf.zip    traefik_waf.zip    haproxy_waf.zip\n                          (published as a GitHub Release)\n```\n\nEach converter is independent, idempotent, and configured exclusively through environment variables (`INPUT_FILE`, `OUTPUT_DIR`). Full reference at [docs/api](https://fabriziosalmi.github.io/patterns/api).\n\n## Repository layout\n\n```text\npatterns/\n├── owasp2json.py            # Pull and parse OWASP CRS into a JSON intermediate\n├── json2nginx.py            # JSON → Nginx (map + if directives)\n├── json2apache.py           # JSON → Apache (ModSecurity SecRule)\n├── json2traefik.py          # JSON → Traefik (middleware TOML)\n├── json2haproxy.py          # JSON → HAProxy (ACL files)\n├── badbots.py               # Public bot lists → per-platform blocklists\n├── import_*_waf.py          # Optional installers for each platform\n├── waf_patterns/            # Generated outputs\n│   ├── nginx/\n│   ├── apache/\n│   ├── traefik/\n│   └── haproxy/\n├── docs/                    # VitePress documentation site\n├── tests/                   # Validation tests for each backend\n└── .github/workflows/       # Daily build + release automation\n```\n\n## Integration in 60 seconds\n\n### Nginx\n\n```nginx\nhttp {\n    include /etc/nginx/waf_patterns/nginx/waf_maps.conf;\n    include /etc/nginx/waf_patterns/nginx/bots.conf;\n}\nserver {\n    include /etc/nginx/waf_patterns/nginx/waf_rules.conf;\n    if ($bad_bot) { return 403; }\n}\n```\n\n### Apache (ModSecurity)\n\n```apache\n\u003cIfModule security2_module\u003e\n    SecRuleEngine On\n    Include /etc/apache2/waf_patterns/apache/*.conf\n\u003c/IfModule\u003e\n```\n\n### Traefik\n\n```yaml\nhttp:\n  routers:\n    app:\n      rule: \"Host(`example.com`)\"\n      service: app\n      middlewares: [waf-protection@file, bot-blocker@file]\n```\n\n### HAProxy\n\n```haproxy\nfrontend http-in\n    bind *:80\n    acl waf_match path,url_dec -m reg -i -f /etc/haproxy/waf.acl\n    acl bad_bot   hdr(User-Agent) -m reg -i -f /etc/haproxy/bots.acl\n    http-request deny deny_status 403 if waf_match || bad_bot\n```\n\nFull guides \u0026mdash; with logging, whitelists, and tuning \u0026mdash; live in the [docs](https://fabriziosalmi.github.io/patterns/).\n\n## Bad-bot example output (Nginx)\n\n```nginx\nmap $http_user_agent $bad_bot {\n    default 0;\n    \"~*AhrefsBot\"  1;\n    \"~*SemrushBot\" 1;\n    \"~*MJ12bot\"    1;\n    \"~*GPTBot\"     1;\n}\n\nif ($bad_bot) { return 403; }\n```\n\nThe default list blocks SEO crawlers, AI training bots, and known scanners while explicitly **allowing** major search engines (Google, Bing, DuckDuckGo, Yandex, Baidu).\n\n## Automation\n\n| Workflow | Schedule | Purpose |\n|----------|----------|---------|\n| [`update_patterns.yml`](.github/workflows/update_patterns.yml) | Daily + manual | Re-fetch CRS, regenerate every backend, publish a release |\n| [`test_nginx.yml`](.github/workflows/test_nginx.yml) | On PR | Validate generated Nginx rules against a live container |\n| [`test_apache_docker.yml`](.github/workflows/test_apache_docker.yml) | On PR | Validate generated Apache rules against ModSecurity in Docker |\n| [`docs.yml`](.github/workflows/docs.yml) | On `docs/` change | Build and deploy the VitePress docs to GitHub Pages |\n\nAll workflows run on **GitHub-hosted runners** (`ubuntu-latest`).\n\n## Documentation\n\nThe full documentation lives at **[fabriziosalmi.github.io/patterns](https://fabriziosalmi.github.io/patterns/)** \u0026mdash; built with [VitePress](https://vitepress.dev/) and deployed automatically.\n\n- [Getting Started](https://fabriziosalmi.github.io/patterns/getting-started)\n- [Nginx](https://fabriziosalmi.github.io/patterns/nginx) \u0026middot; [Apache](https://fabriziosalmi.github.io/patterns/apache) \u0026middot; [Traefik](https://fabriziosalmi.github.io/patterns/traefik) \u0026middot; [HAProxy](https://fabriziosalmi.github.io/patterns/haproxy)\n- [Bad Bot Detection](https://fabriziosalmi.github.io/patterns/badbots)\n- [API \u0026 Scripts Reference](https://fabriziosalmi.github.io/patterns/api)\n\n## Contributing\n\n1. Fork the repository.\n2. Create a feature branch: `git checkout -b feature/your-change`.\n3. Commit and push.\n4. Open a pull request \u0026mdash; the test workflows will run automatically.\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for details and [SECURITY.md](SECURITY.md) for the disclosure policy.\n\n## License\n\nReleased under the [MIT License](LICENSE).\n\n## Resources\n\n- [OWASP Core Rule Set](https://github.com/coreruleset/coreruleset)\n- [ModSecurity](https://modsecurity.org/)\n- [Nginx](https://nginx.org/) \u0026middot; [Apache HTTPD](https://httpd.apache.org/) \u0026middot; [Traefik](https://traefik.io/) \u0026middot; [HAProxy](https://www.haproxy.org/)\n- [ai.robots.txt](https://github.com/ai-robots-txt/ai.robots.txt) \u0026mdash; upstream AI-bot list\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  \u003csub\u003eBuilt and maintained by \u003ca href=\"https://github.com/fabriziosalmi\"\u003eFabrizio Salmi\u003c/a\u003e.\u003c/sub\u003e\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabriziosalmi%2Fpatterns","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabriziosalmi%2Fpatterns","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabriziosalmi%2Fpatterns/lists"}