{"id":30181147,"url":"https://github.com/lubyruffy/tcpdumper","last_synced_at":"2025-08-12T08:07:38.886Z","repository":{"id":300739864,"uuid":"1006874833","full_name":"LubyRuffy/tcpdumper","owner":"LubyRuffy","description":"使用gopacket的reassembly实现的高性能tcp协议分析器（支持组包）","archived":false,"fork":false,"pushed_at":"2025-06-23T11:02:28.000Z","size":37,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-23T11:20:51.503Z","etag":null,"topics":["gopacket","pcap","protocol","tcp"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LubyRuffy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-23T06:05:47.000Z","updated_at":"2025-06-23T11:02:31.000Z","dependencies_parsed_at":"2025-06-23T11:20:54.254Z","dependency_job_id":"cb5fde62-cfba-426d-bede-f54cab4664f7","html_url":"https://github.com/LubyRuffy/tcpdumper","commit_stats":null,"previous_names":["lubyruffy/tcpdumper"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/LubyRuffy/tcpdumper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LubyRuffy%2Ftcpdumper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LubyRuffy%2Ftcpdumper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LubyRuffy%2Ftcpdumper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LubyRuffy%2Ftcpdumper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LubyRuffy","download_url":"https://codeload.github.com/LubyRuffy/tcpdumper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LubyRuffy%2Ftcpdumper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270024697,"owners_count":24514054,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-12T02:00:09.011Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gopacket","pcap","protocol","tcp"],"created_at":"2025-08-12T08:07:03.510Z","updated_at":"2025-08-12T08:07:38.869Z","avatar_url":"https://github.com/LubyRuffy.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TCPDumper\n\nTCPDumper 是一个简单易用的Go语言TCP数据包捕获和协议解析库。它封装了pcap抓包和TCP重组的复杂性，让开发者能够快速扩展自定义的TCP协议处理器。\n\n## 特性\n\n- 🚀 **简单易用** - 只需几行代码即可开始TCP数据包分析\n- 🔧 **高度可扩展** - 轻松添加自定义协议处理器\n- 📦 **丰富示例** - 提供HTTP、DNS等协议处理器示例\n- 🎯 **智能检测** - 基于数据内容的协议自动识别\n- 🔄 **TCP重组** - 自动处理TCP分片和重组\n- 📊 **统计信息** - 实时的数据包和流统计\n- 🎛️ **灵活配置** - 支持实时抓包和pcap文件分析\n\n## 快速开始\n\n### 安装\n\n```bash\ngo get github.com/LubyRuffy/tcpdumper\n```\n\n### 基本使用\n\n```go\npackage main\n\nimport (\n    \"log\"\n    \"time\"\n    \n    \"github.com/LubyRuffy/tcpdumper\"\n)\n\nfunc main() {\n    // 创建简单的TCP捕获器\n    dumper := tcpdumper.NewSimpleDumper()\n\n    // todo: 注册自定义协议处理器\n    \n    // 启动捕获\n    err := dumper.Start()\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer dumper.Stop()\n    \n    // 运行10秒\n    time.Sleep(10 * time.Second)\n    \n    // 获取统计信息\n    packets, streams, errors := dumper.GetStats()\n    log.Printf(\"处理了 %d 个数据包, %d 个TCP流, %d 个错误\", packets, streams, errors)\n}\n```\n\n### 从pcap文件分析\n\n```go\ndumper := tcpdumper.NewFileDumper(\"capture.pcap\")\nerr := dumper.Start()\nif err != nil {\n    log.Fatal(err)\n}\ndumper.Stop() // 文件处理完成后自动停止\n```\n\n### 指定网络接口\n\n```go\ndumper := tcpdumper.NewInterfaceDumper(\"eth0\")\nerr := dumper.Start()\nif err != nil {\n    log.Fatal(err)\n}\ndefer dumper.Stop()\n```\n\n## 自定义协议处理\n\n### 简单协议注册\n\n最简单的方式是基于字符串前缀匹配：\n\n```go\ndumper := tcpdumper.NewSimpleDumper()\n\n// 注册Echo协议（以\"ECHO:\"开头）\ndumper.RegisterSimpleProtocol(\"Echo\", \"ECHO:\", func(ident string) tcpdumper.ProtocolProcessor {\n    return \u0026EchoProcessor{ident: ident}\n})\n```\n\n### 方向敏感协议\n\n支持客户端和服务器不同的协议模式：\n\n```go\n// Redis协议：客户端命令以\"*\"开头，服务器响应以\"+\"开头\ndumper.RegisterPatternProtocol(\"Redis\", \"*\", \"+\", func(ident string) tcpdumper.ProtocolProcessor {\n    return \u0026RedisProcessor{ident: ident}\n})\n```\n\n### 自定义协议检测器\n\n对于复杂的协议检测逻辑：\n\n```go\ntype MyProtocolDetector struct{}\n\nfunc (mpd *MyProtocolDetector) Detect(data []byte, dir reassembly.TCPFlowDirection) int {\n    // 检测特定的二进制头部\n    if len(data) \u003e= 4 \u0026\u0026 data[0] == 0xCA \u0026\u0026 data[1] == 0xFE {\n        return 95 // 高置信度\n    }\n    return 0\n}\n\nfunc (mpd *MyProtocolDetector) Name() string {\n    return \"MyProtocol\"\n}\n\nfunc (mpd *MyProtocolDetector) CreateProcessor(streamInfo tcpdumper.StreamInfo) tcpdumper.ProtocolProcessor {\n    return \u0026MyProtocolProcessor{ident: streamInfo.Ident}\n}\n\n// 注册自定义检测器\ndumper.RegisterProtocolDetector(\u0026MyProtocolDetector{})\n```\n\n### 实现协议处理器\n\n所有协议处理器都需要实现 `ProtocolProcessor` 接口：\n\n```go\ntype MyProtocolProcessor struct {\n    ident string\n}\n\nfunc (mp *MyProtocolProcessor) ProcessData(data []byte, dir reassembly.TCPFlowDirection, start, end bool) error {\n    fmt.Printf(\"MyProtocol/%s [%s]: 处理 %d 字节数据\\n\", mp.ident, dir, len(data))\n    \n    // 在这里实现你的协议解析逻辑\n    // dir 参数表示数据流方向：\n    // - reassembly.TCPDirClientToServer: 客户端到服务器\n    // - reassembly.TCPDirServerToClient: 服务器到客户端\n    \n    return nil\n}\n\nfunc (mp *MyProtocolProcessor) Close() error {\n    fmt.Printf(\"MyProtocol/%s: 连接关闭\\n\", mp.ident)\n    return nil\n}\n\nfunc (mp *MyProtocolProcessor) GetProtocolName() string {\n    return \"MyProtocol\"\n}\n```\n\n## 高级配置\n\n### 自定义捕获选项\n\n```go\noptions := \u0026tcpdumper.CaptureOptions{\n    Interface:   \"eth0\",           // 网络接口\n    PcapFile:    \"\",               // pcap文件路径（为空则实时抓包）\n    SnapLen:     65536,            // 每个数据包的最大捕获长度\n    Promiscuous: true,             // 混杂模式\n    Timeout:     time.Millisecond * 30, // 超时时间（毫秒）\n    BPFFilter:   \"tcp port 80\",    // BPF过滤器\n}\n\ndumper := tcpdumper.NewDumper(options)\n```\n\n### BPF过滤器示例\n\n```go\n// 只捕获HTTP流量\noptions.BPFFilter = \"tcp port 80 or tcp port 443\"\n\n// 只捕获特定IP的流量\noptions.BPFFilter = \"host 192.168.1.100\"\n\n// 组合条件\noptions.BPFFilter = \"tcp and (port 80 or port 443) and host 192.168.1.100\"\n```\n\n## 协议示例\n\nTCPDumper 不内置任何协议处理器，但提供了丰富的示例代码供参考：\n\n### HTTP协议示例\n\n参见 `examples/httpdumper/` 目录：\n\n- 完整的HTTP协议检测和处理实现\n- 支持所有标准HTTP方法（GET, POST, PUT等）\n- 特殊处理CONNECT方法（代理模式）\n- 方向敏感检测（请求 vs 响应）\n\n### DNS协议示例\n\n参见 `examples/dnsdumper/` 目录：\n\n- DNS over TCP协议的完整实现\n- 自动检测DNS消息格式\n- 处理DNS查询和响应\n- 支持标准DNS消息结构\n\n## 协议检测机制\n\n### 置信度系统\n\n协议检测基于置信度（0-100）：\n\n- **0-50**: 低置信度，不会被选中\n- **51-80**: 中等置信度，可能的协议匹配\n- **81-100**: 高置信度，很可能是该协议\n\n### 多协议竞争\n\n当多个协议都能检测到同一数据时：\n\n1. 计算每个协议的置信度\n2. 选择置信度最高的协议\n3. 只有置信度\u003e50才会被选中\n\n## 默认处理器\n\n### 处理未知协议\n\n当TCP流没有匹配到任何已注册的协议时，可以使用默认处理器来处理：\n\n```go\ndumper := tcpdumper.NewSimpleDumper()\n\n// 或者使用自定义默认处理器\ndumper.SetDefaultProcessor(func(ident string) tcpdumper.ProtocolProcessor {\n    return \u0026MyDefaultProcessor{ident: ident}\n})\n```\n\n### 自定义默认处理器\n\n```go\ntype MyDefaultProcessor struct {\n    ident string\n    file  *os.File\n}\n\nfunc (mdp *MyDefaultProcessor) ProcessData(data []byte, dir reassembly.TCPFlowDirection, start, end bool) error {\n    // 将原始数据保存到文件\n    if start \u0026\u0026 mdp.file == nil {\n        var err error\n        mdp.file, err = os.Create(fmt.Sprintf(\"unknown_%s.bin\", \n            strings.ReplaceAll(mdp.ident, \":\", \"_\")))\n        if err != nil {\n            return err\n        }\n    }\n    \n    if mdp.file != nil {\n        mdp.file.Write(data)\n    }\n    \n    return nil\n}\n\nfunc (mdp *MyDefaultProcessor) Close() error {\n    if mdp.file != nil {\n        return mdp.file.Close()\n    }\n    return nil\n}\n\nfunc (mdp *MyDefaultProcessor) GetProtocolName() string {\n    return \"Unknown\"\n}\n```\n\n### 统计信息\n\n启用默认处理器后，`GetStats()`方法会返回额外的统计信息：\n\n```go\npackets, tcpStreams, errors, unknownFlows := dumper.GetStats()\nfmt.Printf(\"统计: %d 包, %d 流, %d 错误, %d 未知协议流\\n\", \n    packets, tcpStreams, errors, unknownFlows)\n```\n\n## API参考\n\n### 主要类型\n\n```go\n// 创建捕获器的便捷函数\nfunc NewSimpleDumper() *TCPDumper\nfunc NewFileDumper(filename string) *TCPDumper  \nfunc NewInterfaceDumper(iface string) *TCPDumper\nfunc NewDumper(options *CaptureOptions) *TCPDumper\n\n// TCPDumper 主要方法\nfunc (td *TCPDumper) Start() error\nfunc (td *TCPDumper) Stop()\nfunc (td *TCPDumper) GetStats() (packets, tcpStreams, errors, unknownFlows uint64)\nfunc (td *TCPDumper) GetRegisteredProtocols() []string\nfunc (td *TCPDumper) RegisterSimpleProtocol(name, pattern string, factory func(string) ProtocolProcessor)\nfunc (td *TCPDumper) RegisterPatternProtocol(name, clientPattern, serverPattern string, factory func(string) ProtocolProcessor)\nfunc (td *TCPDumper) RegisterProtocolDetector(detector ProtocolDetector)\nfunc (td *TCPDumper) SetDefaultProcessor(factory DefaultProcessorFactory)\n```\n\n### 接口定义\n\n```go\ntype ProtocolProcessor interface {\n    ProcessData(data []byte, dir reassembly.TCPFlowDirection, start, end bool) error\n    Close() error\n    GetProtocolName() string\n}\n\ntype ProtocolDetector interface {\n    Detect(data []byte, dir reassembly.TCPFlowDirection) int\n    Name() string\n    \tCreateProcessor(streamInfo StreamInfo) ProtocolProcessor\n}\n\ntype DefaultProcessorFactory func(ident string) ProtocolProcessor\n```\n\n## 性能考虑\n\n- **内存使用**: 自动清理过期的TCP流和IPv4碎片\n- **并发安全**: 协议注册表支持并发访问\n- **零拷贝**: 最小化数据拷贝操作\n- **高效检测**: 基于置信度的快速协议匹配\n\n## 使用场景\n\n- 🔍 **网络流量分析** - 分析网络中的各种协议流量\n- 🛡️ **安全监控** - 检测异常的网络行为\n- 🐛 **网络调试** - 诊断网络连接问题  \n- 📊 **协议统计** - 收集协议使用统计信息\n- 🔬 **协议逆向** - 分析未知的网络协议\n- 🧪 **协议开发** - 测试新的网络协议实现\n\n## 完整示例\n\n```go\npackage main\n\nimport (\n    \"fmt\"\n    \"log\"\n    \"time\"\n    \n    \"github.com/LubyRuffy/tcpdumper\"\n    \"github.com/google/gopacket/reassembly\"\n)\n\n// 自定义协议处理器\ntype TelnetProcessor struct {\n    ident string\n}\n\nfunc (tp *TelnetProcessor) ProcessData(data []byte, dir reassembly.TCPFlowDirection, start, end bool) error {\n    fmt.Printf(\"Telnet/%s [%s]: %s\\n\", tp.ident, dir, string(data))\n    return nil\n}\n\nfunc (tp *TelnetProcessor) Close() error {\n    fmt.Printf(\"Telnet/%s: 连接关闭\\n\", tp.ident)\n    return nil\n}\n\nfunc (tp *TelnetProcessor) GetProtocolName() string {\n    return \"Telnet\"\n}\n\nfunc main() {\n    // 创建捕获器\n    options := \u0026tcpdumper.CaptureOptions{\n        Interface: \"lo0\",\n        BPFFilter: \"tcp\",\n        SnapLen:   1024,\n    }\n    dumper := tcpdumper.NewDumper(options)\n    \n    // 注册自定义Telnet协议\n    dumper.RegisterSimpleProtocol(\"Telnet\", \"login:\", func(ident string) tcpdumper.ProtocolProcessor {\n        return \u0026TelnetProcessor{ident: ident}\n    })\n    \n    // 显示已注册的协议\n    protocols := dumper.GetRegisteredProtocols()\n    fmt.Printf(\"已注册协议: %v\\n\", protocols)\n    fmt.Println(\"默认处理器: 已启用\")\n    \n    // 启动捕获\n    err := dumper.Start()\n    if err != nil {\n        log.Fatal(err)\n    }\n    defer dumper.Stop()\n    \n    // 运行30秒\n    fmt.Println(\"开始捕获TCP流量...\")\n    time.Sleep(30 * time.Second)\n    \n    // 获取统计信息\n    packets, streams, errors, unknownFlows := dumper.GetStats()\n    fmt.Printf(\"统计信息: %d 个数据包, %d 个TCP流, %d 个错误, %d 个未知协议流\\n\", \n        packets, streams, errors, unknownFlows)\n}\n```\n\n## 许可证\n\n本项目采用与主项目相同的许可证。 ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flubyruffy%2Ftcpdumper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flubyruffy%2Ftcpdumper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flubyruffy%2Ftcpdumper/lists"}