{"id":13775260,"url":"https://github.com/slgobinath/pcap-processor","last_synced_at":"2025-04-14T17:30:46.355Z","repository":{"id":89869910,"uuid":"132529917","full_name":"slgobinath/pcap-processor","owner":"slgobinath","description":"Read and process pcap files using this nifty tool","archived":false,"fork":false,"pushed_at":"2018-12-21T13:41:29.000Z","size":40,"stargazers_count":40,"open_issues_count":3,"forks_count":21,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-08-03T17:11:19.312Z","etag":null,"topics":["csv","grpc","http","kafka","pcap","pcap-processor","sink","stream","wisdom"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slgobinath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-05-08T00:07:22.000Z","updated_at":"2024-01-31T17:05:29.000Z","dependencies_parsed_at":"2023-03-06T10:30:16.102Z","dependency_job_id":null,"html_url":"https://github.com/slgobinath/pcap-processor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slgobinath%2Fpcap-processor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slgobinath%2Fpcap-processor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slgobinath%2Fpcap-processor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slgobinath%2Fpcap-processor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slgobinath","download_url":"https://codeload.github.com/slgobinath/pcap-processor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219843506,"owners_count":16556504,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["csv","grpc","http","kafka","pcap","pcap-processor","sink","stream","wisdom"],"created_at":"2024-08-03T17:01:35.991Z","updated_at":"2024-10-15T15:07:21.024Z","avatar_url":"https://github.com/slgobinath.png","language":"Python","readme":"# Pcap Processor\nRead and process pcap files using this nifty tool.\n\nThis tool can read pcap files, process them internally and write them to one or more sinks.\nCurrently there are mappers written for pcap length conversion and protocol normalization.\nI also have written sinks to write the pcap file to console, csv file or http endpoint.\n\n```text\nusage: pcap-processor [-h] [--map {length,protocol}]\n                      [--sink {console,kafka,http,csv,grpc}] [--version]\n                      file [file ...]\n\nRead and process pcap files using this nifty tool.\n\npositional arguments:\n  file                  pcap file to read\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --map {length,protocol}\n                        enable a mapper with the given name. You can use this\n                        option multiple times to enable more than one mappers\n  --sink {console,kafka,http,csv,grpc}\n                        enable a sink with the given name. You can use this\n                        option multiple times to enable more than one sinks\n  --version             show program's version number and exit\n```\n\n## Note\n\nCurrently the `pcap-processor` reads only packets with IP or IPV6 layers. Feel free to modify the code based on your requirement. If you find any enhancement, please send a Pull Request.\n\n## Requirements\n\npcap-reader relies on external command line tool: `tshark` and some Python modules.\n\nInstall `tshark` using the following command in Ubuntu and its derivatives:\n\n```bash\nsudo apt install tshark\n```\n\nInstall Python dependencies using the following command:\n\n```bash\npip3 install -r requirements.txt\n```\n\n## Research Work\nThis tool is developed as part of my research project. If you are using this tool in your research,\nplease cite the following paper:\n\n**Citation:**\n\n```text\nLoganathan, G., Samarabandu, J., \u0026 Wang, X. (2018). Sequence to Sequence Pattern Learning Algorithm for Real-time Anomaly Detection in Network Traffic. In 2018 IEEE Canadian Conference on Electrical \u0026 Computer Engineering (CCECE) (CCECE 2018). Quebec City, Canada.\n```\n\n**BibTex**\n\n```bibtex\n@INPROCEEDINGS{Loga1805:Sequence,\nAUTHOR=\"Gobinath Loganathan and Jagath Samarabandu and Xianbin Wang\",\nTITLE=\"Sequence to Sequence Pattern Learning Algorithm for Real-time Anomaly\nDetection in Network Traffic\",\nBOOKTITLE=\"2018 IEEE Canadian Conference on Electrical \\\u0026 Computer Engineering (CCECE)\n(CCECE 2018)\",\nADDRESS=\"Quebec City, Canada\",\nDAYS=13,\nMONTH=may,\nYEAR=2018,\nKEYWORDS=\"Seq2Seq; Anomaly Detection\",\nABSTRACT=\"Network intrusions can be modeled as anomalies in network traffic in which\nthe expected order of packets and their attributes deviate from regular\ntraffic. Algorithms that predict the next sequence of events based on\nprevious sequences are a promising avenue for detecting such anomalies. In\nthis paper, we present a novel multi-attribute model for predicting a\nnetwork packet sequence based on previous packets using a\nsequence-to-sequence (Seq2Seq) encoder-decoder model. This model is trained\non an attack-free dataset to learn the normal sequence of packets in TCP\nconnections and then it is used to detect anomalous packets in TCP traffic.\nWe show that in DARPA 1999 dataset, the proposed multi-attribute Seq2Seq\nmodel detects anomalous raw TCP packets which are part of intrusions with\n97\\% accuracy. Also, it can detect selected intrusions in real-time with\n100\\% accuracy and outperforms existing algorithms based on recurrent\nneural network models such as LSTM.\"\n}\n```\n\n## Use Cases\n\nRead a pcap file and send all packets to Apache Kafka:\n\n```bash\npython3 -m pcap_processor --sink kafka samples/cicids_2017.pcap\n```\n\nRead a pcap file, map protocols and write them to a CSV file:\n\n```bash\npython3 -m pcap_processor --map protocol --sink csv samples/cicids_2017.pcap\n```\n\nMappers and sinks have their own properties. Please modify them in the relevant `plugins/\u003cfile\u003e.py`.\n\nFor example, to change the output CSV file location, modify the `self.path = \"packets.csv\"` in `pcap_processor/plugins/csv_sink.py`.","funding_links":[],"categories":["\u003ca id=\"f13469c9891173804423be4403b2c4ff\"\u003e\u003c/a\u003epcap"],"sub_categories":["\u003ca id=\"eb49514924c3f4bf2acf6f3a4436af13\"\u003e\u003c/a\u003e未分类"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslgobinath%2Fpcap-processor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslgobinath%2Fpcap-processor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslgobinath%2Fpcap-processor/lists"}