{"id":13775237,"url":"https://github.com/fireeye/brocapi","last_synced_at":"2025-05-11T07:32:13.770Z","repository":{"id":72459779,"uuid":"108190206","full_name":"fireeye/brocapi","owner":"fireeye","description":"Bro PCAP Processing and Tagging API","archived":true,"fork":false,"pushed_at":"2017-11-09T22:49:52.000Z","size":13,"stargazers_count":28,"open_issues_count":3,"forks_count":11,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-17T10:39:40.124Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fireeye.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS"}},"created_at":"2017-10-24T22:22:19.000Z","updated_at":"2024-10-11T22:03:41.000Z","dependencies_parsed_at":"2024-01-17T13:11:59.242Z","dependency_job_id":"1f7c9e18-f86e-497b-a3e1-2560b424a74c","html_url":"https://github.com/fireeye/brocapi","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fireeye%2Fbrocapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fireeye%2Fbrocapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fireeye%2Fbrocapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fireeye%2Fbrocapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fireeye","download_url":"https://codeload.github.com/fireeye/brocapi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253533897,"owners_count":21923515,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-03T17:01:35.683Z","updated_at":"2025-05-11T07:32:13.475Z","avatar_url":"https://github.com/fireeye.png","language":"Python","funding_links":[],"categories":["\u003ca id=\"f13469c9891173804423be4403b2c4ff\"\u003e\u003c/a\u003epcap"],"sub_categories":["\u003ca id=\"eb49514924c3f4bf2acf6f3a4436af13\"\u003e\u003c/a\u003e未分类"],"readme":"## Brocapi\nHTTP API for mass processing PCAPs through Bro IDS, tagging, and submitting the resulting logs to a syslog server.\n\n## Overview\nBrocapi consists of two parts: \n* A Flask API for receiving PCAPS via HTTP POST and queues a job in Redis via RQ\n* An RQ worker that receives the jobs queued from the API to process the PCAPs through Bro and submit the resulting logs to a syslog server.\n\n## Requirements\n### Python Modules\n```\nflask\nrq\nredis\n```\n### System Packages\n```\nBro IDS\nRedis instance running on the same host as the worker\nA Syslog server accepting TCP or UDP connections\n```\n### Optional\n```\ngunicorn\nsupervisor\n```\n\n## Configuration\nBy default Brocapi expects the config to be located at /etc/brocapi.conf\n```\n[main]\n# debug logging\ndebug = off\n# worker log file\nworker_log = /var/log/brocapi/brocapi_worker.log\n# api log file\napi_log = /var/log/brocapi/brocapi_api.log\n\n[bro]\n# location of your Bro binary\nbro_bin = /opt/bro/bin/bro\n# directory to contain job output\nprocessing_dir = /opt/brocapi/jobs/\n# bro logs to ignore completely\nlog_blacklist = [\"loaded_scripts.log\", \"packet_filter.log\", \"reporter.log\"]\n\n[syslog]\n# syslog host\nsyslog_host = 127.0.0.1\n# syslog port\nsyslog_port = 514\n# syslog protocol (tcp/udp)\nsyslog_proto = tcp\n# syslog program prefix (format string). For example if you use \"bro_\" as the program prefix, your HTTP logs will have the syslog program bro_http\nsyslog_prefix = bro25_%s%\n```\n\n## Running Brocapi\nThe API and workers can be run directly i.e.:\n```\n./brocapi.py\n```\nbut there are a few sample configurations for running Brocapi as a service under Systemd:\n* brocapi.service: Runs the flask app under gunicorn\n* brocapi_worker.service: Runs the brocapi worker under supervisor. Included the supervisor config in brocapi_worker.ini\n\n## Submitting a Job\nBrocapi only has one route `/submit/pcap` and it expects a POST request with 1 required and 1 optional parameter:\n* 1 or more PCAPs in an array called 'file'\n* Optional 'tag' parameter which Brocapi will use as the value for the syslog hostname\n\nExample submission using Curl without a job tag:\n```\n{11:38}~/Desktop ➭ curl -k -X POST -F 'file[]=@2017-08-28-Fobos-campaign-Rig-EK-sends-Bunitu.pcap' https://127.0.0.1/submit/pcap\n{\"status\": \"job queued\", \"files\": [\"2017-08-28-Fobos-campaign-Rig-EK-sends-Bunitu.pcap\"], \"tag\": null, \"job_id\": \"9179876e-08cf-4539-8de7-8a8bb3b0dcaf\", \"success\": true}\n```\nExample submissions using Curl with a job tag:\n```\n{11:39}~/Desktop ➭ curl -k -X POST -F 'file[]=@2017-08-28-Fobos-campaign-Rig-EK-sends-Bunitu.pcap' -F 'tag=testing' https://127.0.0.1/submit/pcap\n{\"status\": \"job queued\", \"files\": [\"2017-08-28-Fobos-campaign-Rig-EK-sends-Bunitu.pcap\"], \"tag\": \"testing\", \"job_id\": \"507965ab-6511-4cd4-9542-4671eb140f92\", \"success\": true}%\n```\n\n### Returned data:\n```\n{\n  \"status\": \"job queued\",                                 ## Status of the job. Right now the only value is \"job queued\" since we're not async and waiting on the full status\n  \"files\": [\n    \"2017-08-28-Fobos-campaign-Rig-EK-sends-Bunitu.pcap\"  ## An array of the pcaps that were submitted\n  ],\n  \"tag\": \"testing\",                                       ## The tag that was supplied to mark the job\n  \"job_id\": \"507965ab-6511-4cd4-9542-4671eb140f92\",       ## UUID for the job\n  \"success\": true                                         ## Success or failure\n}\n```\n## Job Directories\nOnce a job is received, the API will create the following example directory structure for the job:\n```\njobs # Configured jobs directory\n└── fe5f53b3-474d-4cb7-8ece-a2786f841af7 # UUID for the job\n    ├── logs # Directory for all the logs\n    │   ├── bro # Directory containing all the Bro logs\n    │   │   ├── capture_loss.log\n    │   │   ├── conn.log\n    │   │   ├── dhcp.log\n    │   │   ├── dns.log\n    │   │   ├── files.log\n    │   │   ├── http.log\n    │   │   ├── loaded_scripts.log\n    │   │   ├── packet_filter.log\n    │   │   ├── ssl.log\n    │   │   ├── stats.log\n    │   │   ├── weird.log\n    │   │   └── x509.log\n    │   └── syslog # Not used right now\n    └── pcaps # Directory containing the submitted PCAPS\n        ├── 1725aa89-2f9e-5a44-88da-6bce278e77d3.pcap\n        ├── 1ec8ca5f-66dd-5200-9cf2-235638ef13f9.pcap\n        ├── 20a3341d-8134-5136-a03c-cb8c3d3fc5be.pcap\n        ├── 20cf04e4-a9ef-5415-9f17-bf4c80214c33.pcap\n        └── 2f506083-8a84-57c4-8d37-9304157b0899.pcap\n```\n\n## Worker Activity\nOnce the API queues a job into Redis via RQ, a worker will pick up the job and perform the following actions\n* Iterate through the supplied PCAPS and invoke Bro with the following paramters: `bro -C -r \u003cpcap\u003e local`\n* If any logs were generated, create a connection to the configured syslog server\n* Iterate through each generated log type and submit each line to the configured syslog server\n\n## TODO\n* Make config location configurable\n* Make log_blacklist configurable. It exists in the config, but isn't honored by the worker\n* Make the Redis host configurable\n* Support moving logs on the host post-job for setups where a log sender might be monitoring a directory\n* Add a route for retrieving logs, pcaps, status, etc of previously submitted jobs","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffireeye%2Fbrocapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffireeye%2Fbrocapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffireeye%2Fbrocapi/lists"}