{"id":19522356,"url":"https://github.com/hxsecurity/dongtai_usb","last_synced_at":"2025-04-26T09:32:08.416Z","repository":{"id":219386165,"uuid":"616410016","full_name":"HXSecurity/Dongtai_USB","owner":"HXSecurity","description":null,"archived":false,"fork":false,"pushed_at":"2023-08-30T04:15:35.000Z","size":410,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-04T10:37:14.310Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HXSecurity.png","metadata":{"files":{"readme":"docs/README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-20T10:36:58.000Z","updated_at":"2023-04-24T03:58:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"68b44787-a8eb-4788-8973-336bf8703843","html_url":"https://github.com/HXSecurity/Dongtai_USB","commit_stats":null,"previous_names":["hxsecurity/dongtai_usb"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HXSecurity%2FDongtai_USB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HXSecurity%2FDongtai_USB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HXSecurity%2FDongtai_USB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HXSecurity%2FDongtai_USB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HXSecurity","download_url":"https://codeload.github.com/HXSecurity/Dongtai_USB/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250967253,"owners_count":21515565,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T00:38:37.307Z","updated_at":"2025-04-26T09:32:08.407Z","avatar_url":"https://github.com/HXSecurity.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Dongtai_USB\n\u003cbr /\u003e\n\n### 转换器配置文件(放在当前目录下)：config-tutorial.ini.example \n```\nip: xray访问地址白名单,默认不需要修改\niast_url：iast地址\ndast_token： iast对应dast_token\ntype: 扫描器类型（xray）\nxray_url：商业版xray地址\nxray_token： 商业版xray-token\n```\n\n### 启动\n```\nmv config-tutorial.ini.example config-tutorial.ini\ndocker-compose up -d\n添加代理：IP:10802\n```\n\u003cbr /\u003e\n\u003cbr /\u003e\n\n## 开发配置\n### 数据上报流程: \n```\n用户 ==\u003e 浏览器代理 ==\u003e mitmproxy ==\u003e xray ==\u003e dongtai_usb ==\u003e 洞态IAST\n```\n![Alt text](image-1.png)\n\n\n1. 需要添加两个 header 响应头\n```\nDt-Request-Id\ndt-mark-header\n```\n2. 通过 mitmproxy 自动添加 dt-mark-header 响应头\n```\nflow.request.headers[\"dt-mark-header\"] = uuid.uuid4().hex\n```\n3. 通过 洞态agent 自动添加 Dt-Request-Id  响应头\n```\ndt-request-id : \u003cagent_id\u003e.\u003cuuid\u003e\n```\n\n\n4. 发送给IAST的数据格式，可参考如下结构体Response，可直接调用\nservice.Client(Response) 发送数据给洞态iast,\n```\ntype Response struct {\n\tVulName         string            `json:\"vul_name\"`\n\tDetail          string            `json:\"detail\"`\n\tVulLevel        string            `json:\"vul_level\"`\n\tUrls            []string          `json:\"urls\"`\n\tPayload         string            `json:\"payload\"`\n\tCreateTime      int64             `json:\"create_time\"`\n\tVulType         string            `json:\"vul_type\"`\n\tRequestMessages []RequestMessages `json:\"request_messages\"`\n\tTarget          string            `json:\"target\"`\n\tDtUUIDID        []string          `json:\"dt_uuid_id\"`\n\tAgentID         []string          `json:\"agent_id\"`\n\tDongtaiVulType  []string          `json:\"dongtai_vul_type\"`\n\tDtmark          []string          `json:\"dt_mark\"`\n\tDastTag         string            `json:\"dast_tag\"`\n}\ntype RequestMessages struct {\n\tRequest  string `json:\"request\"`\n\tResponse string `json:\"response\"`\n}\n\ntype Target struct {\n\tParams []struct {\n\t\tPath     []string `json:\"path\"`\n\t\tPosition string   `json:\"position\"`\n\t} `json:\"params\"`\n\tURL string `json:\"url\"`\n}\n```\n\n5. Response 结构体详解\n```\n{\n    \"vul_name\": \"\",#漏洞名 格式为 target+漏洞类型\n    \"detail\":\"\", #漏洞详情\n    \"vul_level\": \"HIGH\", #HIGH,MEDIUM,LOW,NOTE 漏洞等级，对应现在洞态的4个等级\n    \"urls\":[\"\"],# 黑盒扫描发送的多个 url 地址\n    \"payload\":\"\", #  黑盒扫描触发漏洞的 payload, 可为空\n    \"create_time\":1679020853, # 时间戳(秒)\n    \"vul_type\":\"\",#黑盒扫描的漏洞类型\n    \"request_messages\":[{ # 一组扫描对应的所有请求和响应信息\n        \"request\":\"\",\n        \"response\":\"\"\n      }\n    ],\n    #以下为dongtai对接相关信息。\n    \"dt_mark\": [\"\"], # dt-mark-header 的值\n    \"target\":\"\", # 原始请求地址\n    \"dt_uuid_id\":[\"\"], # 需要在 dt-request-id 响应头拆分出来\n    \"agent_id\":[\"\"], # 需要在 dt-request-id 响应头拆分出来\n    \"dongtai_vul_type\":[\"\"],# 洞态的漏洞类型, 多个类型，为空数组即对应所有调用链漏洞\n    \"dast_tag\":\"\", # 所集成的黑盒扫描器标识\n}\n```\n6. 上报消息示例\n```\n{\n\t\"vul_name\": \"http://192.168.1.37:8001/xxe/Digester/vuln xxe\",\n\t\"detail\": \"在http://192.168.1.37:8001/xxe/Digester/vuln发现了xxe\",\n\t\"vul_level\": \"MEDIUM\",\n\t\"urls\": [\"/xxe/Digester/vuln\"],\n\t\"payload\": \"\\u003c?xml version=\\\"1.0\\\"?\\u003e\\u003c!DOCTYPE ANY [\\u003c!ENTITY content SYSTEM \\\"XRAY_RURL\\\"\\u003e]\\u003e\\u003ca\\u003e\\u0026content;\\u003c/a\\u003e\",\n\t\"create_time\": 1687334492,\n\t\"vul_type\": \"xxe\",\n\t\"request_messages\": [{\n\t\t\"request\": \"POST /xxe/Digester/vuln HTTP/1.1\\r\\nHost: 192.168.1.37:8001\\r\\nUser-Agent: Xray_Test\\r\\nContent-Length: 123\\r\\nAccept: */*\\r\\nContent-Type: application/xml\\r\\nCookie: XSRF-TOKEN=b8442e83-8a47-4703-9b4c-b0e74dc214f8; JSESSIONID=60EB9A6A04AAF9416210F3951788EA81\\r\\nDt-Dast: Xray\\r\\nDt-Mark-Header: b9c9ddd59de04c479577851db7dd0242\\r\\nXray: x\\r\\nAccept-Encoding: gzip\\r\\n\\r\\n\\u003c?xml version=\\\"1.0\\\"?\\u003e\\u003c!DOCTYPE ANY [\\u003c!ENTITY content SYSTEM \\\"http://177.7.0.11:8091/i/4d6816/1pqo/b9t8/\\\"\\u003e]\\u003e\\u003ca\\u003e\\u0026content;\\u003c/a\\u003e\",\n\t\t\"response\": \"HTTP/1.1 200 \\r\\nCache-Control: no-cache, no-store, max-age=0, must-revalidate\\r\\nContent-Length: 22\\r\\nContent-Type: text/plain;charset=UTF-8\\r\\nDate: Wed, 21 Jun 2023 08:01:31 GMT\\r\\nDongtai: v1.10.0\\r\\nDt-Request-Id: 17.0ce4ae9dca114fd084a2070cf96e99cd\\r\\nExpires: 0\\r\\nPragma: no-cache\\r\\nX-Application-Context: application\\r\\nX-Content-Type-Options: nosniff\\r\\nX-Frame-Options: DENY\\r\\nX-Xss-Protection: 1; mode=block\\r\\n\\r\\nDigester xxe vuln code\"\n\t}],\n\t\"target\": \"http://192.168.1.37:8001/xxe/Digester/vuln\",\n\t\"dt_uuid_id\": [\"0ce4ae9dca114fd084a2070cf96e99cd\"],\n\t\"agent_id\": [\"17\"],\n\t\"dongtai_vul_type\": [\"xxe\"],\n\t\"dt_mark\": [\"b9c9ddd59de04c479577851db7dd0242\"],\n\t\"dast_tag\": \"Xray\"\n}\n```\n\n\n## 如何开发一个新的黑盒扫描器\n```\n1. 在dongtai_usb/目录下创建一个新的文件夹，文件夹名字为黑盒扫描器的名字\n2. 在新建的文件夹下创建三个子目录,可参考xray目录\n\t1. dongtai_usb/xxx/engine/  # 数据处理转换 代码\n\t2. dongtai_usb/xxx/model/ # 请求结构体 代码\n\t3. dongtai_usb/xxx/request/ # 接收或拉取请求实现 代码\n\n3. 漏洞类型可使用map对应，参考: dongtai_usb/xray/model/vultype.go\n\t1. Vultype 为漏洞类型命名\n\t2. VulLevel 为漏洞等级命名\n```\n### 漏洞类型等级对应关系\n```\nfunc Vultype() map[string]string {\n\treturn map[string]string{\n\t\t//xray漏洞类型     //洞态漏洞类型\n\t\t\"xss\":            \"reflected-xss\",\n\t\t\"sqldet\":         \"sql-injection\",\n\t\t\"cmd-injection\":  \"cmd-injection\",\n\t\t\"path-traversal\": \"path-traversal\",\n\t\t\"xxe\":            \"xxe\",\n\t\t\"ssrf\":           \"ssrf\",\n\t\t\"brute-force\":    \"crypto-bad-ciphers\",\n\t\t\"redirect\":       \"unvalidated-redirect\",\n\t}\n}\nfunc VulLevel() map[string]string {\n\treturn map[string]string{\n\t\t//xray漏洞名字     //洞态漏洞等级\n\t\t\"xss\":            \"MEDIUM\",\n\t\t\"sqldet\":         \"HIGH\",\n\t\t\"cmd-injection\":  \"HIGH\",\n\t\t\"path-traversal\": \"HIGH\",\n\t\t\"xxe\":            \"MEDIUM\",\n\t\t\"ssrf\":           \"HIGH\",\n\t\t\"brute-force\":    \"LOW\",\n\t\t\"redirect\":       \"LOW\",\n\t}\n}\n```\n\n洞态支持的漏洞类型可参照: https://github.com/HXSecurity/DongTai/blob/main/static/data/vul_strategy.json 此文件中的 `vul_type` 即为漏洞类型\n\n### 运行方式为两个场景\n```\n推流模式(webhook)：即黑盒扫描器主动推送数据给dongtai_usb\n拉流模式(cron): 即dongtai_usb主动从黑盒扫描器定时拉取数据\n```\n\n![Alt text](image-2.png)\n### 开发完成后在main方法添加启用即可，如xray\n```\n推流模式(webhook):\nrouter.POST(\"/v1/xray\", USB_Xray.Xray)\n\n拉流模式(cron):\nusb.Cron(\"xray\", USB_Xray.Xray_cron)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhxsecurity%2Fdongtai_usb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhxsecurity%2Fdongtai_usb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhxsecurity%2Fdongtai_usb/lists"}