{"id":13540129,"url":"https://github.com/chengdedeng/waf","last_synced_at":"2025-04-02T06:32:06.848Z","repository":{"id":45714939,"uuid":"93134078","full_name":"chengdedeng/waf","owner":"chengdedeng","description":":vertical_traffic_light:Web Application Firewall or API Gateway(应用防火墙/API网关)","archived":false,"fork":false,"pushed_at":"2024-01-17T08:56:29.000Z","size":1409,"stargazers_count":692,"open_issues_count":0,"forks_count":208,"subscribers_count":49,"default_branch":"master","last_synced_at":"2025-01-07T19:49:24.966Z","etag":null,"topics":["firewall","http-proxy","littleproxy","security","waf"],"latest_commit_sha":null,"homepage":"https://github.com/chengdedeng/waf","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chengdedeng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-02T06:24:50.000Z","updated_at":"2024-12-21T14:28:47.000Z","dependencies_parsed_at":"2024-08-01T09:23:56.529Z","dependency_job_id":"62e2c962-f3df-4d15-95a4-3a4ece8f792c","html_url":"https://github.com/chengdedeng/waf","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengdedeng%2Fwaf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengdedeng%2Fwaf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengdedeng%2Fwaf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chengdedeng%2Fwaf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chengdedeng","download_url":"https://codeload.github.com/chengdedeng/waf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246768374,"owners_count":20830654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["firewall","http-proxy","littleproxy","security","waf"],"created_at":"2024-08-01T09:01:41.036Z","updated_at":"2025-04-02T06:32:04.422Z","avatar_url":"https://github.com/chengdedeng.png","language":"Java","funding_links":[],"categories":["\u003ca id=\"946d766c6a0fb23b480ff59d4029ec71\"\u003e\u003c/a\u003e防护\u0026\u0026Defense","Java","\u003ca id=\"0abd611fc3e9a4d9744865ca6e47a6b2\"\u003e\u003c/a\u003e工具","Java (504)","API网关"],"sub_categories":["\u003ca id=\"784ea32a3f4edde1cd424b58b17e7269\"\u003e\u003c/a\u003eWAF"],"readme":"[![Join the chat at https://gitter.im/chengdedeng/waf](https://badges.gitter.im/chengdedeng/waf.svg)](https://gitter.im/chengdedeng/waf?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n[![MIT Licence](https://badges.frapsoft.com/os/mit/mit.svg)](https://opensource.org/licenses/mit-license.php)\n[![LICENSE](https://img.shields.io/badge/license-Anti%20996-blue.svg)](https://github.com/996icu/996.ICU/blob/master/LICENSE)\n\n\u003e WAF是使用Java开发的API Gateway，由于WAF构建在开源代理[LittleProxy](https://github.com/adamfisk/LittleProxy)之上，所以说WAF底层使用的是[Netty](https://github.com/netty/netty)。\n\n### 特性\n1. 安全拦截，支持各种分析检测，支持脚本（沙箱）；\n2. 流控/CC防护，支持IP粒度，可扩展；\n3. HTTP代理，支持\"hijacking\" HTTPS connection using \"Man in the Middle\" style attack；\n4. URL Rewrite；\n5. HTTP Redirect；\n6. Socks5；\n7. 集中式配置；\n8. 自定义协议转化，HTTP-\u003eDUBBO，HTTP-\u003eGRPC等（可导入swagger接口文档）；\n\n### Quick Start\n\n##### 编译:\n```\n mvn package\n```\n\n##### 运行:\n由于使用[appassembler-maven-plugin](http://www.mojohaus.org/appassembler/appassembler-maven-plugin/usage-jsw.html)\n打成了符合[JSW](https://wrapper.tanukisoftware.com/doc/english/download.jsp)规范的包,所以解压target目录下的Zip文件,\n然后在bin目录下运行对应平台的脚本,以Linux为例:\n\n```\nbin/waf { console | start | stop | restart | status | dump }\n```\n\n##### 配置:\n2.0开始配置分为基础配置和应用配置，基础配置就是Classpath下的waf.properties、admin.properties、application.properties、cluster.properties；\n应用配置则需要通过接口进行设置，waf目前还没有一套简易的UI来进行设置，不过集成了swagger，可以通过swagger ui界面来配置。2.0相较于1.0，配置集中化之后的好处在于配置修改\n不再需要停机重新加载，规则随时可以添加、禁用删除。配置中心目前zookeeper是稳定的，atomix自研的配置中心目前还不完善，建议别开启atomix的spi实现。尽管没有UI，但是接口做了\n详细的校验，大家可以放心设置，不大会出现配置参数设置错误导致的系统崩溃。\n\n\n### 架构\nHTTP Proxy选择了基于[Netty](https://netty.io/)研发的[LittleProxy](https://github.com/adamfisk/LittleProxy),\nLittleProxy是[LANTERN](https://getlantern.org/)的维护者发起的开源项目,是一款非常优秀的Java HTTP Proxy.\n关于Loadbalance,WAF有两种模式可以供选择,一种基于Proxy Chain,另一种是基于HostResolver.Proxy Chain是把目标机的映射交给\n下游的Proxy,而HostResolver则是WAF自身完成映射.需要特别注意的是,Proxy Chain中如果存在多Proxy是不会负载均衡的,只有前一个不可用时才会用下一个.\n\n**HttpRequestFilterChain** 和 **HttpResponseFilterChain** 责任链,分别对进来和出去的数据进行拦截分析.Request拦截又分为黑白名单两种,Response拦截主要给输出的数据进行安全加固.在Request的拦截规则方面,我参考了[loveshell/ngx_lua_waf](https://github.com/loveshell/ngx_lua_waf).\n\n更多技术详情请移步个人[Java版WAF技术细节](http://www.yangguo.info/2017/06/06/Java%E7%89%88WAF%E5%AE%9E%E7%8E%B0/#more)\n[HttpProxy研发心得](http://www.yangguo.info/2017/11/13/HttpProxy%E7%A0%94%E5%8F%91%E5%BF%83%E5%BE%97/#more)\n\n\n### 性能\n\n##### 测试目的\nNginx的性能是有目共睹的,WAF既然作为一个HTTP Proxy,所以需要跟Nginx对比一下,看看性能的差距有多大.\n\n因为目的是要压出中间Proxy的性能极限,所以后端服务性能要非常高,至少要比中间Proxy性能好,所以选用了Nginx模拟后端服务.\n为了减少网络开销对测试影响,所有的测试都是在一台机器上完成的.\n\n\n##### 测试基准:\n1.AB-\u003eNginx_Proxy-\u003eNginx_AS\n\n2.AB-\u003eWAF-\u003eNginx_AS\n\n3.ab -k -c 100 -n 1000000 目标地址(HTTP长链)\n\n4.ab -c 100 -n 1000000 目标地址(HTTP短链)\n\n\n##### JDK版本\n```\njava version \"1.8.0_131\"\nJava(TM) SE Runtime Environment (build 1.8.0_131-b11)\nJava HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)\n```\n\n##### WAF JVM配置:\n```\nwrapper.java.additional.1=-server\nwrapper.java.additional.2=-Xms2048m\nwrapper.java.additional.3=-Xmx2048m\nwrapper.java.additional.4=-Xmn800m\nwrapper.java.additional.5=-XX:+UseG1GC\nwrapper.java.additional.6=-Xloggc:/tmp/log/gc.log\nwrapper.java.additional.7=-XX:+HeapDumpOnOutOfMemoryError\nwrapper.java.additional.8=-XX:+PrintGCDetails\nwrapper.java.additional.9=-XX:+PrintGCTimeStamps\nwrapper.java.additional.10=-XX:+PreserveFramePointer\n```\n\n##### WAF基础配置:\n```\n#on表示waf支持loadbalance,需要配置upstream.properties,与waf.proxy.chain和waf.mitm互斥\nwaf.lb=on\n#设置重试间隔时间，默认10秒\nwaf.lb.fail_timeout=10\n#是否路由到waf下游的proxy,与waf.proxy.lb互斥\nwaf.chain=off\n#waf下游的proxy,多个用\",\"分隔.注意只有前一个不可用,才会用下一个,下游proxy不会负载均衡\nwaf.chain.servers=127.0.0.1:4321\n#是否启用TLS,与waf.mitm互斥\nwaf.tls=off\n#是否HTTPS开启中间人拦截,与waf.tls和waf.proxy.lb互斥\nwaf.mitm=off\n#接收者线程数,如果系统只有一个服务端port需要监听,则BossGroup线程组线程数设置为 1。\n#https://stackoverflow.com/questions/22280916/do-we-need-more-than-a-single-thread-for-boss-group\nwaf.acceptorThreads=1\n#处理client请求的工作线程数\nwaf.clientToProxyWorkerThreads=100\n#处理proxy与后端服务器的工作线程数\nwaf.proxyToServerWorkerThreads=100\n#waf服务器端口\nwaf.serverPort=9091\n#是否开启Socks5支持\nwaf.ss=off\nwaf.ss.server.host=127.0.0.1\nwaf.ss.server.port=1080\n#The timeout (in seconds) for auto-closing idle connections.\nwaf.idleConnectionTimeout=70\n```\n\n##### 服务器/虚拟机(测试机)配置:\n\n```\n4  Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz\n```\n\n\n#### 结果:\n\n#### CPU(id基本在10以内)\n\n```\n%Cpu0  : 49.8 us, 33.7 sy,  0.0 ni,  6.1 id,  0.0 wa,  0.0 hi, 10.4 si,  0.0 st\n%Cpu1  : 48.0 us, 33.9 sy,  0.0 ni,  7.4 id,  0.0 wa,  0.0 hi, 10.7 si,  0.0 st\n%Cpu2  : 49.8 us, 33.0 sy,  0.0 ni,  7.4 id,  0.0 wa,  0.0 hi,  9.8 si,  0.0 st\n%Cpu3  : 48.8 us, 31.5 sy,  0.0 ni,  8.5 id,  0.0 wa,  0.0 hi, 11.2 si,  0.0 st\n```\n\n#### QPS\n\n测试场景|测试条件|QPS\n-------|-------|-------\nAB-\u003eNginx_AS|HTTP长链|64815\nAB-\u003eNginx_AS|HTTP短链|6174\nAB-\u003eNginx_Proxy-\u003eNginx_AS|HTTP长链|16924\nAB-\u003eNginx_Proxy-\u003eNginx_AS|HTTP短链|13137\nAB-\u003eWAF-\u003eNginx_AS|HTTP长链|5566\nAB-\u003eWAF-\u003eNginx_AS|HTTP短链|5559\n\n\n#### 火焰图:\n\ngithub不支持火焰图显示,[点击下载源文件](https://github.com/chengdedeng/waf/blob/master/doc/flamegraph.svg).\n\n![](https://github.com/chengdedeng/waf/blob/master/doc/framegraph.png)\n\n\n#### 常见问题\n1. 开启TLS or MITM后,会在项目的目录下生成waf_cert证书,TLS会自动下发证书,MITM需要手动加入证书,信任之后就可以正常工作了.\n2. `waf.proxy.lb`和`waf.proxy.mitm`,`waf.tls`和`waf.proxy.mitm`,`waf.proxy.chain`和`waf.proxy.lb`两两之间只能开启其中之一.\n3. 如果只是HTTP或者HTTPS抓包,可以关闭所有的安全拦截.\n4. 如果Gateway前面还有loadbalance，例如Nginx，Nginx-\u003eGateway一定要用HTTP1.1，否者会报`upstream prematurely closed connection while reading upstream`。出现该问题的原因\n可以仔细阅读[HttpFilterAdapterImpl.java](https://github.com/chengdedeng/waf/blob/master/src/main/java/info/yangguo/waf/HttpFilterAdapterImpl.java)中方法\n**proxyToServerRequestSending**的注释。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchengdedeng%2Fwaf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchengdedeng%2Fwaf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchengdedeng%2Fwaf/lists"}