{"id":15523717,"url":"https://github.com/arloor/proxyme","last_synced_at":"2025-04-23T07:16:15.031Z","repository":{"id":105556455,"uuid":"144016656","full_name":"arloor/proxyme","owner":"arloor","description":"基于java NIO的http代理，支持https","archived":false,"fork":false,"pushed_at":"2022-05-18T16:57:11.000Z","size":733,"stargazers_count":16,"open_issues_count":0,"forks_count":16,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-23T07:16:04.619Z","etag":null,"topics":["http","proxy"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arloor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-08T13:20:32.000Z","updated_at":"2025-03-06T01:46:52.000Z","dependencies_parsed_at":"2024-03-25T05:45:19.265Z","dependency_job_id":null,"html_url":"https://github.com/arloor/proxyme","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arloor%2Fproxyme","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arloor%2Fproxyme/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arloor%2Fproxyme/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arloor%2Fproxyme/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arloor","download_url":"https://codeload.github.com/arloor/proxyme/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250386841,"owners_count":21422040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["http","proxy"],"created_at":"2024-10-02T10:46:40.595Z","updated_at":"2025-04-23T07:16:15.021Z","avatar_url":"https://github.com/arloor.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e 这是我早期学习java NIO时做的项目，现在基本都是通过Netty来做NIO，不会有人直接把java NIO用在生产环境了。所以这个项目对大多数人的意义应该是用于学习java NIO的example。当然不是说了解java NIO没有意义，了解了java NIO，你才能知道Netty是怎么工作的。想要完整功能的httpproxy可以看看这个项目[HttpProxy](https://github.com/arloor/HttpProxy)\n\n# proxyme 一个http代理\n\n使用java NIO的http代理。支持https。\n\n因为墙的原因，这个代理不会处理域名中有`google`、`youtube`、`facebook`的请求。\n\n之前也打算做过这个东西，结果做出来的有点缺陷（现在想可能是selector中锁的问题，忘记了）。这大概隔了半年，这个项目的http代理功能实现了。\n\n## 运行日志\n\n```\n11:23:08.883 [main] INFO com.arloor.proxyme.HttpProxyBootStrap - 在8080端口启动了代理服务\n11:23:12.208 [localSelector] INFO com.arloor.proxyme.LocalSelector - 接收浏览器连接: /127.0.0.1:50317\n11:23:12.210 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 请求—— CONNECT cn.bing.com:443 HTTP/1.1\n11:23:12.291 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 创建远程连接: cn.bing.com/202.89.233.100:443\n11:23:12.291 [remoteSlector] INFO com.arloor.proxyme.RemoteSelector - 注册remoteChannel到remoteSelector。remoteChannel: cn.bing.com/202.89.233.100:443\n11:23:12.298 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 发送请求517 --\u003ecn.bing.com/202.89.233.100:443\n11:23:12.365 [remoteSlector] INFO com.arloor.proxyme.ChannalBridge - 接收响应2720 \u003c--cn.bing.com/202.89.233.100:443\n11:23:12.365 [remoteSlector] INFO com.arloor.proxyme.ChannalBridge - 接收响应1360 \u003c--cn.bing.com/202.89.233.100:443\n11:23:12.366 [remoteSlector] INFO com.arloor.proxyme.ChannalBridge - 接收响应1360 \u003c--cn.bing.com/202.89.233.100:443\n11:23:12.369 [remoteSlector] INFO com.arloor.proxyme.ChannalBridge - 接收响应1360 \u003c--cn.bing.com/202.89.233.100:443\n11:23:12.369 [remoteSlector] INFO com.arloor.proxyme.ChannalBridge - 接收响应1 \u003c--cn.bing.com/202.89.233.100:443\n11:23:12.378 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 发送请求93 --\u003ecn.bing.com/202.89.233.100:443\n11:23:12.382 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 发送请求1022 --\u003ecn.bing.com/202.89.233.100:443\n...\n...\n11:23:13.281 [localSelector] INFO com.arloor.proxyme.LocalSelector - 接收浏览器连接: /127.0.0.1:50319\n11:23:13.282 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 请求—— GET http://s.cn.bing.net/th?id=OSA.xiipvhS2Pp2bEg\u0026w=80\u0026h=80\u0026c=8\u0026rs=1\u0026pid=SatAns HTTP/1.1\n11:23:13.382 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 创建远程连接: s.cn.bing.net/112.84.133.11:80\n11:23:13.383 [remoteSlector] INFO com.arloor.proxyme.RemoteSelector - 注册remoteChannel到remoteSelector。remoteChannel: s.cn.bing.net/112.84.133.11:80\n11:23:13.383 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 发送请求340 --\u003es.cn.bing.net/112.84.133.11:80\n11:23:13.383 [localSelector] INFO com.arloor.proxyme.ChannalBridge - 发送请求409 --\u003es.cn.bing.net/112.84.133.11:80\n```\n\n## 性能与内存\n\n占用cpu不到1%\n\n内存最大35m（不含jvm自身）。GC次数和时间很少\n\n总的来说，性能可以了吧。\n\n## 思路\n\n两个线程，每个线程一个selector。\n\nlocalSelector线程，负责接收本地浏览器的连接请求和读写浏览器到代理的socketChannel\n\nremoteSelector线程，负责读写web服务器到代理的socketChannel。\n\nChannelBridge类,持有localSocketChannel和remoteSocketChannel。职责是处理请求和响应，并转发。\n\nRequestHeader类，职责是格式化请求行和请求头。\n\n## 实现中的注意点\n\n首先是健壮性！每一个try块都是很重要的！都解决了一个问题\n\n其次是锁的问题：\n\nselector.select()会占有锁，channel.register(selector)需要持有同样的锁。\n\n如果调用上面的两个方法的语句在两个线程中，会让channel.regiter等很久很久，导致响应难以及时得到。具体分析可以看看netty是怎么做的，见[从register和accept的锁竞争问题到netty的nioEventLoop设计](https://www.arloor.com/posts/netty/select-register-nioeventloop/)\n\n而在实现中，这是一个生产者消费者问题。localSelector线程根据本地浏览器请求产生了一个从代理到web服务器的remoteChannel。而remoteSelector要接收这个remoteChannel,这也就是消费了。\n\n很自然的，避免上面锁等待最好的方法：localSelector生成remoteChannel，将其放入队列。remoteSelector线程从队列中取。再结合selector.wakeup()使其从阻塞中返回，可以快速地接收（register）这个remoteChannel。\n\n这两点，就是最最重要的两点了。\n\n另外还有，因为代理而需要改变的请求头了，参见`com.arloor.proxyme.RequestHeader.reform()`方法。\n\n最后，https代理实现中的坑。http代理传输的内容是明文，字节肯定大于0，而https传输的字节可能小于0。因为这个，传输https数据的bybebuff时，要特意指定bytebuff的limit为实际大小。\n\n还有一个小问题，向remoteChannel 写的时候，有时候会写0个字节，原因是底层tcp缓冲满了，我的处理是等0.1秒，再继续传。当然设置OP_WRITE这个监听选项的目的就是处理这种情况。\n\nhttp代理不神秘。\n\n## 命令行参数\n\n可以添加两个命令行参数： `host=xxxx port=8080`\n\n如果不设置这两个参数： 本地代理将使用`InetAddress.getLocalhost()`的ip和`8080`的端口。\n\n注意，一台电脑会有好几个ip地址。如果使用127.0.0.1，则该代理只可以在本机上使用。如果使用局域网地址（最常见的是192.168.x.x），并合理配置jdk的防火墙权限，则可以在局域网中使用该代理。\n\n那么如何把该代理部署到云服务器上？如下：\n\n```\n[root@VM_26_36_centos ~]ifconfig\neth0: flags=4163\u003cUP,BROADCAST,RUNNING,MULTICAST\u003e  mtu 1500\n        inet 10.154.26.36  netmask 255.255.192.0  broadcast 10.154.63.255\n        ether 52:54:00:b5:bb:6a  txqueuelen 1000  (Ethernet)\n        RX packets 41677295  bytes 5458697312 (5.0 GiB)\n        RX errors 0  dropped 0  overruns 0  frame 0\n        TX packets 41351236  bytes 5660742157 (5.2 GiB)\n        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0\n\nlo: flags=73\u003cUP,LOOPBACK,RUNNING\u003e  mtu 65536\n        inet 127.0.0.1  netmask 255.0.0.0\n        loop  txqueuelen 1000  (Local Loopback)\n        RX packets 3370828  bytes 195574819 (186.5 MiB)\n        RX errors 0  dropped 0  overruns 0  frame 0\n        TX packets 3370828  bytes 195574819 (186.5 MiB)\n        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0\n\n[root@VM_26_36_centos ~]# java -jar proxyme.jar host=10.154.26.36\n15:09:03.245 [main] WARN HttpProxyBootStrap - 提示：允许携带两个命令行参数 host=xxx port=1234\n15:09:03.337 [main] INFO LocalSelector - 在10.154.26.36:8080端口启动了代理服务。注意可能非127.0.0.1\n```\n\n先ifconfig找到自己的云服务器内网地址（其实和局域网地址一个意思），然后在启动的命令行参数中，增加`host=内网地址`，进行启动。当然还需要配置防火墙。配好之后就可以连上服务器使用代理了。\n\n不过，不可以用来翻墙。。\n\n## 可以改进的地方\n\n对channel的读写可以加入更多的线程来进行。\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farloor%2Fproxyme","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farloor%2Fproxyme","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farloor%2Fproxyme/lists"}