{"id":19769409,"url":"https://github.com/goplus/bpl","last_synced_at":"2025-04-30T17:31:09.271Z","repository":{"id":37076958,"uuid":"185920899","full_name":"goplus/bpl","owner":"goplus","description":"Binary Processing Language","archived":false,"fork":false,"pushed_at":"2022-06-17T18:51:01.000Z","size":473,"stargazers_count":153,"open_issues_count":0,"forks_count":32,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-04-06T03:23:08.893Z","etag":null,"topics":["binary-parser","bpl","go","golang","language","text-processing"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/goplus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-05-10T04:53:00.000Z","updated_at":"2024-12-22T23:58:58.000Z","dependencies_parsed_at":"2022-06-24T18:20:23.219Z","dependency_job_id":null,"html_url":"https://github.com/goplus/bpl","commit_stats":null,"previous_names":["qiniu/bpl"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goplus%2Fbpl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goplus%2Fbpl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goplus%2Fbpl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goplus%2Fbpl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/goplus","download_url":"https://codeload.github.com/goplus/bpl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251751127,"owners_count":21637866,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary-parser","bpl","go","golang","language","text-processing"],"created_at":"2024-11-12T04:43:03.373Z","updated_at":"2025-04-30T17:31:08.992Z","avatar_url":"https://github.com/goplus.png","language":"Go","readme":"BPL - Binary Processing Language\n============\n\n[![Build Status](https://github.com/goplus/bpl/actions/workflows/go.yml/badge.svg)](https://github.com/goplus/bpl/actions/workflows/go.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/goplus/bpl)](https://goreportcard.com/report/github.com/goplus/bpl)\n[![GitHub release](https://img.shields.io/github/v/tag/goplus/bpl.svg?label=release)](https://github.com/goplus/bpl/releases)\n[![Coverage Status](https://codecov.io/gh/goplus/bpl/branch/master/graph/badge.svg)](https://codecov.io/gh/goplus/bpl)\n[![GoDoc](https://pkg.go.dev/badge/github.com/goplus/bpl.svg)](https://pkg.go.dev/mod/github.com/goplus/bpl)\n\n## 快速入门\n\n了解 BPL 最快的方式是学习 qbpl 和 qbplproxy 两个实用程序：\n\n### qbpl\n\nqbpl 可用来分析任意的文件格式。使用方法如下：\n\n```\nqbpl [-p \u003cprotocol\u003e.bpl -o \u003coutput\u003e.log] \u003cfile\u003e\n```\n\n多数情况下，你不需要指定 `-p \u003cprotocol\u003e.bpl` 参数，我们根据文件后缀来确定应该使用何种 protocol 来解析这个文件。例如：\n\n```\nqbpl 1.gif\n```\n\n不过为了让 qbpl 能够找到所有的 protocols，我们需要先安装：\n\n```\nmake install # 这将将所有的bpl文件拷贝到 ~/.qbpl/formats/\n```\n\n### qbplproxy\n\nqbplproxy 可用来分析服务器和客户端之间的网络包。它通过代理要分析的服务，让客户端请求自己来分析请求包和返回包。使用方式如下：\n\n```\nqbplproxy -h \u003clistenIp:port\u003e -b \u003cbackendIp:port\u003e [-p \u003cprotocol\u003e.bpl -f \u003cfilter\u003e -o \u003coutput\u003e.log]\n```\n\n其中，`\u003clistenIp:port\u003e` 是 qbplproxy 自身监听的IP和端口，`\u003cbackendIp:port\u003e` 是原始的服务。`-f \u003cfilter\u003e` 是过滤条件，这个条件通过 BPL_FILTER 全局变量传递到 bpl 中。\n\n多数情况下，你不需要指定 `-p \u003cprotocol\u003e.bpl` 参数，qbplproxy 程序可以根据你监听的端口来猜测网络协议。例如：\n\n```\nmongod --port 37017\nqbplproxy -h localhost:27017 -b localhost:37017\n```\n\n我们会依据端口 27017 知道你要分析的是 mongodb 的网络协议。\n\n\n## BPL 文法\n\n请参见 [BPL 文法](README_BPL.md)。\n\n\n## 网络协议研究\n\n### RTMP 协议\n\n格式描述：\n\n* [rtmp.bpl](formats/rtmp.bpl)\n\n测试：\n\n1) 启动一个 rtmp server，让其监听 1936 端口（而不是默认的 1935 端口）。比如我们可以用 [node-rtsp-rtmp-server](https://github.com/iizukanao/node-rtsp-rtmp-server)：\n\n```\ngit clone git@github.com:iizukanao/node-rtsp-rtmp-server.git\ncd node-rtsp-rtmp-server\n修改 config.coffee，将：\n  * rtmpServerPort: 1935 改为 rtmpServerPort: 1936；\n  * serverPort: 80 改为 serverPort: 8080（这样就不用 sudo 来运行了）\ncoffee server.coffee\n```\n\n2) 启动 qbplproxy：\n\n```\nqbplproxy -h localhost:1935 -b localhost:1936 -p formats/rtmp.bpl | tee rtmp.log\n```\n\n3) 推流：\n\n```\nffmpeg -re -i test.m4v -c:v copy -c:a copy -f flv rtmp://localhost/live/123\n```\n\n4) 播流：\n\n在 Mac 下可以考虑用 VLC Player，打开网址 rtmp://localhost/live/123 进行播放即可。\n\n5) 选择性查看\n\n有时候我们并不希望看到所有的信息，rtmp.bpl 支持以 flashVer 作为过滤条件。如：\n\n```\nqbplproxy -f 'flashVer=LNX 9,0,124,2' -h localhost:1935 -b localhost:1936 -p formats/rtmp.bpl | tee \u003coutput\u003e.log\n```\n\n或者我们直接用 reqMode(用来区分是推流publish还是播流play) 来过滤。如：\n\n```\nqbplproxy -f 'reqMode=play' -h localhost:1935 -b localhost:1936 -p formats/rtmp.bpl | tee \u003coutput\u003e.log\n```\n\n这样就可以只捕获 VLC Player 的播流过程了。\n\n当然，其实还有一个不用过滤条件的办法：就是让推流直接推到 rtmp server，但是播流请求发到 qbplproxy。\n\n\n### FLV 协议\n\n格式描述：\n\n* [flv.bpl](formats/flv.bpl)\n\n测试：\n\n1) 启动一个 rtmp/flv server，让其监听 1935/8135 端口。\n\n2) 启动 qbplproxy：\n\n```\nqbplproxy -h localhost:8888 -b localhost:8135 -p formats/flv.bpl | tee flv.log\n```\n\n3) 推流：\n\n```\nffmpeg -re -i test.m4v -c:v copy -c:a copy -f flv rtmp://localhost/live/123\n```\n\n4) 播流：\n\n在 Mac 下可以考虑用 VLC Player，打开网址 http://localhost:8888/live/123.flv 进行播放即可。\n\n\n### WebRTC 协议\n\n格式描述：\n\n* [webrtc.bpl](formats/webrtc.bpl)\n\n\n### MongoDB 协议\n\n格式描述：\n\n* [mongo.bpl](formats/mongo.bpl)\n\n测试：\n\n1) 启动 MongoDB，让其监听 37017 端口（而不是默认的 27017 端口）：\n\n```\n./mongod --port 37017 --dbpath ~/data/db\n```\n\n2) 启动 qbplproxy：\n\n```\nqbplproxy -h localhost:27017 -b localhost:37017 -p formats/mongo.bpl | tee mongo.log\n```\n\n3) 使用 MongoDB，比如通过 mongo shell 操作：\n\n```\n./mongo\n```\n\n## 文件格式研究\n\n### MongoDB binlog 格式\n\nTODO\n\n### MySQL binlog 格式\n\nTODO\n\n### HLS TS 格式\n\n格式描述：\n\n* [ts.bpl](formats/ts.bpl)\n\n测试：TODO\n\n### FLV 格式\n\n格式描述：\n\n* [flv.bpl](formats/flv.bpl)\n\n测试：TODO\n\n### MP4 格式\n\n格式描述：\n\n* [mp4.bpl](formats/mp4.bpl)\n\n测试：\n\n```\nqbpl -p formats/mp4.bpl \u003cexample\u003e.mp4\n```\n\n### GIF 格式\n\n格式描述：\n\n* [gif.bpl](formats/gif.bpl)\n\n测试：\n\n```\nqbpl -p formats/gif.bpl formats/1.gif\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoplus%2Fbpl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoplus%2Fbpl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoplus%2Fbpl/lists"}