{"id":16366384,"url":"https://github.com/centminmod/centminmod-csf-lfd-parser","last_synced_at":"2025-07-31T18:06:41.667Z","repository":{"id":181531082,"uuid":"619015921","full_name":"centminmod/centminmod-csf-lfd-parser","owner":"centminmod","description":null,"archived":false,"fork":false,"pushed_at":"2023-09-14T17:19:31.000Z","size":79,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-31T01:28:56.541Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/centminmod.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-26T02:18:34.000Z","updated_at":"2024-05-22T01:21:15.000Z","dependencies_parsed_at":"2024-06-21T17:57:41.667Z","dependency_job_id":"f8439ba0-1d66-45a5-8e53-63fa19731696","html_url":"https://github.com/centminmod/centminmod-csf-lfd-parser","commit_stats":{"total_commits":30,"total_committers":2,"mean_commits":15.0,"dds":"0.033333333333333326","last_synced_commit":"0d5d2bac0c48b0e41f73ea48551b69e117c8d729"},"previous_names":["centminmod/centminmod-csf-lfd-parser"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centminmod%2Fcentminmod-csf-lfd-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centminmod%2Fcentminmod-csf-lfd-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centminmod%2Fcentminmod-csf-lfd-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/centminmod%2Fcentminmod-csf-lfd-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/centminmod","download_url":"https://codeload.github.com/centminmod/centminmod-csf-lfd-parser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239827776,"owners_count":19703749,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T02:46:18.728Z","updated_at":"2025-02-20T11:27:30.810Z","avatar_url":"https://github.com/centminmod.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"![GitHub last commit](https://img.shields.io/github/last-commit/centminmod/centminmod-csf-lfd-parser) ![GitHub contributors](https://img.shields.io/github/contributors/centminmod/centminmod-csf-lfd-parser) ![GitHub Repo stars](https://img.shields.io/github/stars/centminmod/centminmod-csf-lfd-parser) ![GitHub watchers](https://img.shields.io/github/watchers/centminmod/centminmod-csf-lfd-parser) ![GitHub Sponsors](https://img.shields.io/github/sponsors/centminmod) ![GitHub top language](https://img.shields.io/github/languages/top/centminmod/centminmod-csf-lfd-parser) ![GitHub language count](https://img.shields.io/github/languages/count/centminmod/centminmod-csf-lfd-parser)\n\nCentmin Mod CSF LFD Log Parser\n\nFour versions - shell script, Python, Golang and Rust versions - see [benchmarks](#benchmarks).\n\n* [`lfd-parser.sh`](#shell-script-version) - requires both MaxMind GeoLite2 ASN database at `/usr/share/GeoIP/GeoLite2-ASN.mmdb` and `/usr/local/nginx-dep/bin/mmdblookup` be available.\n* [`lfd-parser.py`](#python-version) - requires both MaxMind GeoLite2 ASN database at `/usr/share/GeoIP/GeoLite2-ASN.mmdb` and `/usr/local/nginx-dep/bin/mmdblookup` be available.\n* [`lfd-parser.go`](#golang-version) - requires only that MaxMind GeoLite2 ASN database at `/usr/share/GeoIP/GeoLite2-ASN.mmdb` be available as it uses `geoip2-golang` instead of `mmdblookup`. Supports [filtering](#filtering) options for `--ip`, `--asn` and `--info`.\n* [`lfd-parser.rs`](#rust-version) - requires `Cargo.toml` file. Supports [filtering](#filtering-rust) options for `-i`, `-a`, `-d` and `-n`. Including [Standalone binary builds](#alternative-standalone-rust-binary). You can download a standalone Rust binary for `lfd_parser` on releases page https://github.com/centminmod/centminmod-csf-lfd-parser/releases.\n\nAll four versions parses the CSF LFD `lfd.log` log for timestamp, IP address and type but additionally does an optional IP ASN number/organization lookup if it detects local MaxMind GeoLite2 ASN database being installed. The local MaxMind GeoLite 2 ASN database will be installed and available when:\n\n1. Centmin Mod persistent config `/etc/centminmod/custom_config.inc` set with `MM_LICENSE_KEY='YOUR_MAXMIND_LICENSEKEY'` and `MM_CSF_SRC='y'`. Where `YOUR_MAXMIND_LICENSEKEY` is your Maxmind GeoLite2 database API Key you sign up for at https://www.maxmind.com/en/geolite2/signup\n2. Centmin Mod persistent config `/etc/centminmod/custom_config.inc` set with `NGINX_GEOIPTWOLITE='y'` before Nginx install or Nginx recompiles (centmin.sh menu option 4). The local MaxMind GeoLite 2 ASN database will automatically update over time.\n\nWhen `NGINX_GEOIPTWOLITE='y'`, `MM_LICENSE_KEY='YOUR_MAXMIND_LICENSEKEY'` and `MM_CSF_SRC='y'` are set in Centmin Mod persistent config `/etc/centminmod/custom_config.inc`, then `mmdblookup` command will be available at `/usr/local/nginx-dep/bin/mmdblookup` and MaxMind GeoLite2 ASN database at `/usr/share/GeoIP/GeoLite2-ASN.mmdb`. The `lfd-parser.sh` script can then take advantage of having a local MaxMind GeoLite2 ASN database to query and lookup an IP addresses' ASN info.\n\n```\nls -lah /usr/share/GeoIP/ | grep mmdb\n-rw-r--r--    1 root root 7.8M Mar 20 17:24 GeoLite2-ASN.mmdb\n-rw-r--r--    1 root root  70M Mar 20 17:25 GeoLite2-City.mmdb\n-rw-r--r--    1 root root 5.6M Mar 20 17:26 GeoLite2-Country.mmdb\n```\n\n```\n/usr/local/nginx-dep/bin/mmdblookup --version\n\n  mmdblookup version 1.7.1\n```\n```\n/usr/local/nginx-dep/bin/mmdblookup --file /usr/share/GeoIP/GeoLite2-ASN.mmdb --ip 187.1.178.101 | sed 's/\u003c[^\u003e]*\u003e//g; s/^[[:space:]]*//'\n\n{\n\"autonomous_system_number\": \n21574 \n\"autonomous_system_organization\": \n\"Century Telecom Ltda\" \n}\n```\n\n# Shell Script Version\n\nExample output from command:\n\n```\n./lfd-parser.sh\n```\n\n```json\n[\n  {\n    \"timestamp\": \"Mar 26 02:37:08\",\n    \"ip\": \"92.205.40.41\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 21499,\n    \"asn_org\": \"Host Europe GmbH\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n\nCorresponding `lfd.log` entries\n\n```\negrep 'Blocked in csf|SSH login' /var/log/lfd.log | tail -3\nMar 26 02:37:08 inc2 lfd[299494]: (sshd) Failed SSH login from 92.205.40.41 (DE/Germany/-): 5 in the last 3600 secs - *Blocked in csf* [LF_SSHD]\nMar 26 02:44:29 inc2 lfd[299629]: (sshd) Failed SSH login from 117.132.192.31 (CN/China/-): 5 in the last 3600 secs - *Blocked in csf* [LF_SSHD]\nMar 26 02:44:29 inc2 lfd[299630]: 117.132.192.31 (CN/China/-), 5 distributed sshd attacks on account [root] in the last 3600 secs - *Blocked in csf* [LF_DISTATTACK]\n```\n\n```bash\n./lfd-parser.sh \u003e parsed.log\n```\n```\ncat parsed.log | jq -r '.[] | .ip' | sort | uniq -c | sort -rn | head -n10\n      2 85.152.30.138\n      2 81.22.233.170\n      2 80.251.216.10\n      2 79.9.37.49\n      2 67.205.174.220\n      2 43.128.233.179\n      2 41.72.219.102\n      2 207.249.96.147\n```\n```\ncat parsed.log | jq -r '.[] | \"\\(.ip) \\(.asn_number) \\(.asn_org) \\(.info)\"' | sort | uniq -c | sort -rn | head -n10\n      1 98.159.98.85 396073 MAJESTIC-HOSTING-01 LF_DISTATTACK\n      1 97.65.33.11 3549 LVLT-3549 LF_SSHD\n      1 95.85.27.201 14061 DIGITALOCEAN-ASN LF_SSHD\n      1 95.85.124.113 20661 State Company of Electro Communications Turkmentelecom LF_SSHD\n      1 95.232.253.35 3269 Telecom Italia LF_SSHD\n      1 95.152.60.98 12389 Rostelecom LF_DISTATTACK\n      1 95.106.174.126 12389 Rostelecom LF_SSHD\n      1 94.153.212.78 15895 Kyivstar PJSC LF_SSHD\n```\n\n# Python Version\n\nThere's also a Python version `lfd-parser.py` for Python 3.6+ and above that will parse the logs and process the ASN info way faster than `lfd-parser.sh`.\n\n```\ntime python3 lfd-parser.py \u003e parsed-python.log\n\nreal    0m1.087s\nuser    0m0.445s\nsys     0m0.774s\n\ntime ./lfd-parser.sh \u003e parsed.log\n\nreal    1m27.451s\nuser    2m40.641s\nsys     0m11.483s\n```\n\n```\ncat parsed-python.log | jq -r '.[] | .ip' | sort | uniq -c | sort -rn | head -n10\n      2 85.152.30.138\n      2 81.22.233.170\n      2 80.251.216.10\n      2 79.9.37.49\n      2 67.205.174.220\n      2 43.128.233.179\n      2 41.72.219.102\n      2 207.249.96.147\n```\n```\ncat parsed-python.log | jq -r '.[] | \"\\(.ip) \\(.asn_number) \\(.asn_org) \\(.info)\"' | sort | uniq -c | sort -rn | head -n10\n      1 98.159.98.85 396073 MAJESTIC-HOSTING-01 LF_DISTATTACK\n      1 97.65.33.11 3549 LVLT-3549 LF_SSHD\n      1 95.85.27.201 14061 DIGITALOCEAN-ASN LF_SSHD\n      1 95.85.124.113 20661 State Company of Electro Communications Turkmentelecom LF_SSHD\n      1 95.232.253.35 3269 Telecom Italia LF_SSHD\n      1 95.152.60.98 12389 Rostelecom LF_DISTATTACK\n      1 95.106.174.126 12389 Rostelecom LF_SSHD\n      1 94.153.212.78 15895 Kyivstar PJSC LF_SSHD\n```\n\n# Golang Version\n\nCentmin Mod install GO via `addons/golang.sh` and then exist SSH session and relogin\n\n```\n/usr/local/src/centminmod/addons/golang.sh install\n```\n\n```\nmkdir lfd-parser\ncd lfd-parser\ngo install github.com/oschwald/geoip2-golang@latest\ngo mod init lfd-parser\ngo get github.com/oschwald/geoip2-golang\n# build it\ngo build -ldflags=\"-s -w\" -o lfd-parser lfd-parser.go\n```\nThen you can run `lfd-parser`\n\n```\ntime ./lfd-parser --p /var/log/lfd.log\n[\n  {\n    \"timestamp\": \"Mar 26 04:35:32\",\n    \"ip\": \"36.248.12.38\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 4837,\n    \"asn_org\": \"CHINA UNICOM China169 Backbone\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 04:52:33\",\n    \"ip\": \"36.112.171.51\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 4847,\n    \"asn_org\": \"China Networks Inter-Exchange\",\n    \"info\": \"LF_DISTATTACK\"\n  },\n  {\n    \"timestamp\": \"Mar 26 05:24:13\",\n    \"ip\": \"54.37.196.181\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 16276,\n    \"asn_org\": \"OVH SAS\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 05:27:54\",\n    \"ip\": \"210.114.1.46\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 4766,\n    \"asn_org\": \"Korea Telecom\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 05:30:14\",\n    \"ip\": \"155.248.233.18\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 31898,\n    \"asn_org\": \"ORACLE-BMC-31898\",\n    \"info\": \"LF_SSHD\"\n  }\n]\n\nreal    0m0.003s\nuser    0m0.001s\nsys     0m0.001s\n```\n\nTimed comparison for `lfd-parser.py` vs `lfd-parser.sh` vs `lfd-parser` (`lfd-parser.go`)\n\n```\ntime python3 lfd-parser.py /var/log/lfd.log-20230326.gz \u003e parsed-python.log\n\nreal    0m1.088s\nuser    0m0.482s\nsys     0m0.735s\n\ntime ./lfd-parser.sh /var/log/lfd.log-20230326.gz \u003e parsed.log\n\nreal    1m29.289s\nuser    2m40.106s\nsys     0m12.970s\n\ntime ./lfd-parser --p /var/log/lfd.log-20230326.gz \u003e parsed-golang.log\n\nreal    0m0.022s\nuser    0m0.021s\nsys     0m0.002s\n```\n\n```\n/usr/bin/time --format='real: %es user: %Us sys: %Ss cpu: %P maxmem: %M KB cswaits: %w' python3 lfd-parser.py /var/log/lfd.log-20230326.gz \u003e parsed-python.log\n\nreal: 1.13s user: 0.48s sys: 0.78s cpu: 112% maxmem: 14520 KB cswaits: 3434\n\n/usr/bin/time --format='real: %es user: %Us sys: %Ss cpu: %P maxmem: %M KB cswaits: %w' ./lfd-parser.sh /var/log/lfd.log-20230326.gz \u003e parsed.log\n\nreal: 86.57s user: 159.33s sys: 11.05s cpu: 196% maxmem: 18120 KB cswaits: 66934\n\n/usr/bin/time --format='real: %es user: %Us sys: %Ss cpu: %P maxmem: %M KB cswaits: %w' ./lfd-parser --p /var/log/lfd.log-20230326.gz \u003e parsed-golang.log\n\nreal: 0.03s user: 0.03s sys: 0.00s cpu: 105% maxmem: 15256 KB cswaits: 114\n```\n\n```\ncat parsed-golang.log | jq -r '.[] | \"\\(.ip) \\(.asn_number) \\(.asn_org) \\(.info)\"' | sort | uniq -c | sort -rn | head -n10\n\n      1 98.159.98.85 396073 MAJESTIC-HOSTING-01 LF_DISTATTACK\n      1 97.65.33.11 3549 LVLT-3549 LF_SSHD\n      1 95.85.27.201 14061 DIGITALOCEAN-ASN LF_SSHD\n      1 95.85.124.113 20661 State Company of Electro Communications Turkmentelecom LF_SSHD\n      1 95.232.253.35 3269 Telecom Italia LF_SSHD\n      1 95.152.60.98 12389 Rostelecom LF_DISTATTACK\n      1 95.106.174.126 12389 Rostelecom LF_SSHD\n      1 94.153.212.78 15895 Kyivstar PJSC LF_SSHD\n```\n\n## Filtering\n\nThe Golang version `lfd-parser` also supports filtering by:\n\n* `--ip` - filter by IP Address. Passing multiple flag instances - equivalent of `OR` filter.\n* `--asn` - filter by ASN Number. Passing multiple flag instances - equivalent of `OR` filter.\n* `--info` - filter by info field i.e. `LF_SSHD`, `LF_DISTATTACK`. Passing multiple flag instances - equivalent of `OR` filter.\n\n```\n./lfd-parser --p /var/log/lfd.log-20230326.gz --ip 117.132.192.31\n[\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\nSupport multiple flag instances `--ip 117.132.192.31 --ip 2.59.62.229`\n```\n./lfd-parser --p /var/log/lfd.log-20230326.gz --ip 117.132.192.31 --ip 2.59.62.229\n[\n  {\n    \"timestamp\": \"Mar 26 02:35:49\",\n    \"ip\": \"2.59.62.229\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 63023,\n    \"asn_org\": \"AS-GLOBALTELEHOST\",\n    \"info\": \"LF_DISTATTACK\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n```\n./lfd-parser --p /var/log/lfd.log-20230326.gz --asn 9808\n[\n  {\n    \"timestamp\": \"Mar 22 04:11:58\",\n    \"ip\": \"120.210.206.146\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n\n```\nSupport multiple flag instances `--asn 9808 --asn 9318`\n```\n./lfd-parser --p /var/log/lfd.log-20230326.gz --asn 9808 --asn 9318\n[\n  {\n    \"timestamp\": \"Mar 19 06:37:34\",\n    \"ip\": \"110.11.234.8\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9318,\n    \"asn_org\": \"SK Broadband Co Ltd\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n```\n./lfd-parser --p /var/log/lfd.log-20230326.gz --info LF_DISTATTACK\n[\n  {\n    \"timestamp\": \"Mar 26 02:35:49\",\n    \"ip\": \"2.59.62.229\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 63023,\n    \"asn_org\": \"AS-GLOBALTELEHOST\",\n    \"info\": \"LF_DISTATTACK\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n\n# Rust Version\n\nYou can download a standalone Rust binary for `lfd_parser` on releases page https://github.com/centminmod/centminmod-csf-lfd-parser/releases.\n\n```\nmkdir -p /home/rusttmp\nchmod 1777 /home/rusttmp\nexport TMPDIR=/home/rusttmp\n# install Rust via rustup can be uninstalled via \n# rustup self uninstall\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\nsource $HOME/.cargo/env\ncd /home\ncargo new lfd_parser\ncd lfd_parser\n```\n\nIn directory `lfd_parser` create or edit existing `Cargo.toml` file with:\n\n```\n[package]\nname = \"lfd_parser\"\nversion = \"0.1.1\"\nedition = \"2021\"\n\n[dependencies]\nmaxminddb = \"0.21.0\"\nregex = \"1.5.5\"\nserde = \"1.0.136\"\nserde_json = \"1.0.64\"\nclap = \"3.0.14\"\nflate2 = \"1.0.22\"\nserde_derive = \"1.0.136\"\n```\n\nReplace the contents of the `src/main.rs` file with the Rust code below found in `lfd-parsers.rs`:\n\n\nBuild and run Cargo project \n\nIn debug / development mode\n\n```\ncargo run\n```\n\nIn release - this will create an optimized binary in the `target/release` directory like `./target/release/lfd_parser`.\n\n```\ncargo build --release\n```\n\nResulting binary `/target/release/lfd_parser`\n\n```\nls -lah ./target/release/lfd_parser\n-rwxr-xr-x 2 root root 6.6M Mar 27 00:48 ./target/release/lfd_parser\n```\n\nDependencies for built binary are system specific.\n\n```\nldd ./target/release/lfd_parser\n        linux-vdso.so.1 (0x00007ffddf9f9000)\n        libc.so.6 =\u003e /lib64/libc.so.6 (0x00007f82f6af0000)\n        /lib64/ld-linux-x86-64.so.2 (0x00007f82f72c9000)\n        libpthread.so.0 =\u003e /lib64/libpthread.so.0 (0x00007f82f68d0000)\n        libgcc_s.so.1 =\u003e /lib64/libgcc_s.so.1 (0x00007f82f66b8000)\n        libdl.so.2 =\u003e /lib64/libdl.so.2 (0x00007f82f64b4000)\n```\n\n## Help Info\n\nHelp info\n\n```\n./target/release/lfd_parser --help\nLog Analyzer \n\nUSAGE:\n    lfd_parser [OPTIONS]\n\nOPTIONS:\n    -a \u003casn\u003e...         Filter by ASN number\n    -d \u003cdb_path\u003e        Path to the GeoLite2 database [default: /usr/share/GeoIP/GeoLite2-ASN.mmdb]\n    -h, --help          Print help information\n    -i \u003cip\u003e...          Filter by IP address\n    -n \u003cinfo\u003e...        Filter by Info\n    -p \u003cpath\u003e           Path to the log file [default: /var/log/lfd.log]\n```\n\nThen to run the built binary\n\n```\n./target/release/lfd_parser -p /var/log/lfd.log\n```\n\n## Alternative Standalone Rust Binary\n\nYou can download a standalone Rust binary for `lfd_parser` on releases page https://github.com/centminmod/centminmod-csf-lfd-parser/releases.\n\n### How to build standalone Rust binary\n\n```\ncd /home/lfd_parser\nrustup target add x86_64-unknown-linux-musl\ncargo build --release --target x86_64-unknown-linux-musl\ncp ./target/x86_64-unknown-linux-musl/release/lfd_parser /usr/local/bin/\nstrip /usr/local/bin/lfd_parser\n```\nNow can access binary from `/usr/local/bin/lfd_parser`\n\n```\n/usr/local/bin/lfd_parser --help\nLog Analyzer \n\nUSAGE:\n    lfd_parser [OPTIONS]\n\nOPTIONS:\n    -a \u003casn\u003e...         Filter by ASN number\n    -d \u003cdb_path\u003e        Path to the GeoLite2 database [default: /usr/share/GeoIP/GeoLite2-ASN.mmdb]\n    -h, --help          Print help information\n    -i \u003cip\u003e...          Filter by IP address\n    -n \u003cinfo\u003e...        Filter by Info\n    -p \u003cpath\u003e           Path to the log file [default: /var/log/lfd.log]\n```\nNo dependencies\n\n```\nldd /usr/local/bin/lfd_parser\n        statically linked\n```\n\n```\ntime /usr/local/bin/lfd_parser -p /var/log/lfd.log-20230326.gz\n```\n\n## Filtering Rust\n\nExample commands\n```\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -i 117.132.192.31\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -i 117.132.192.31 -i 2.59.62.229\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -a 9808\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -a 9808 -a 9318\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -n LF_DISTATTACK\n```\nExample outputs\n```\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -i 117.132.192.31\n[\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n```\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -i 117.132.192.31 -i 2.59.62.229\n[\n  {\n    \"timestamp\": \"Mar 26 02:35:49\",\n    \"ip\": \"2.59.62.229\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 63023,\n    \"asn_org\": \"AS-GLOBALTELEHOST\",\n    \"info\": \"LF_DISTATTACK\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n```\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -a 9808\n[\n  {\n    \"timestamp\": \"Mar 22 04:11:58\",\n    \"ip\": \"120.210.206.146\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n```\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -a 9808 -a 9318\n[\n  {\n    \"timestamp\": \"Mar 19 06:37:34\",\n    \"ip\": \"110.11.234.8\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9318,\n    \"asn_org\": \"SK Broadband Co Ltd\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_SSHD\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n```\n./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz -n LF_DISTATTACK\n\n[\n  {\n    \"timestamp\": \"Mar 26 02:35:49\",\n    \"ip\": \"2.59.62.229\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 63023,\n    \"asn_org\": \"AS-GLOBALTELEHOST\",\n    \"info\": \"LF_DISTATTACK\"\n  },\n  {\n    \"timestamp\": \"Mar 26 02:44:29\",\n    \"ip\": \"117.132.192.31\",\n    \"type\": \"Blocked in csf\",\n    \"asn_number\": 9808,\n    \"asn_org\": \"China Mobile Communications Group Co., Ltd.\",\n    \"info\": \"LF_DISTATTACK\"\n  }\n]\n```\n\n## Benchmarks\n\nParsing CSF Firewall's LFD log file at `/var/log/lfd.log-20230326.gz`\n\n```\nls -lah /var/log/lfd.log*\n-rw------- 1 root root 28K Mar 27 00:53 /var/log/lfd.log\n-rw------- 1 root root 24K Mar 26 02:44 /var/log/lfd.log-20230326.gz\n```\n\nTimed comparison for `lfd-parser.py` vs `lfd-parser.sh` vs `lfd-parser` (`lfd-parser.go`) vs `/target/release/lfd_parser` (rust)\n\n```\ntime python3 lfd-parser.py /var/log/lfd.log-20230326.gz \u003e parsed-python.log\n\nreal    0m1.003s\nuser    0m0.495s\nsys     0m0.647s\n\ntime ./lfd-parser.sh /var/log/lfd.log-20230326.gz \u003e parsed.log\n\nreal    1m26.528s\nuser    2m39.696s\nsys     0m10.917s\n\ntime ./lfd-parser --p /var/log/lfd.log-20230326.gz \u003e parsed-golang.log\n\nreal    0m0.023s\nuser    0m0.021s\nsys     0m0.004s\n\ntime ./target/release/lfd_parser -p /var/log/lfd.log-20230326.gz \u003e parsed-rust.log\n\nreal    0m0.009s\nuser    0m0.004s\nsys     0m0.005s\n```\n\n| Language | Script/Executable                | Speed-up Factor | Real Time  | User Time  | System Time |\n|----------|----------------------------------|-----------------|------------|------------|-------------|\n| Python   | `python3 lfd-parser.py`          | 86.29x          | 0m1.003s   | 0m0.495s   | 0m0.647s    |\n| Shell    | `./lfd-parser.sh`                | 1.00x           | 1m26.528s  | 2m39.696s  | 0m10.917s   |\n| Golang   | `./lfd-parser`                   | 3762.09x        | 0m0.023s   | 0m0.021s   | 0m0.004s    |\n| Rust     | `./target/release/lfd_parser`    | 9614.22x        | 0m0.009s   | 0m0.004s   | 0m0.005s    |\n\nQuerying the `parsed-rust.log`\n\n```\ncat parsed-rust.log | jq -r '.[] | \"\\(.ip) \\(.asn_number) \\(.asn_org) \\(.info)\"' | sort | uniq -c | sort -rn | head -n10\n      1 98.159.98.85 396073 MAJESTIC-HOSTING-01 LF_DISTATTACK\n      1 97.65.33.11 3549 LVLT-3549 LF_SSHD\n      1 95.85.27.201 14061 DIGITALOCEAN-ASN LF_SSHD\n      1 95.85.124.113 20661 State Company of Electro Communications Turkmentelecom LF_SSHD\n      1 95.232.253.35 3269 Telecom Italia LF_SSHD\n      1 95.152.60.98 12389 Rostelecom LF_DISTATTACK\n      1 95.106.174.126 12389 Rostelecom LF_SSHD\n      1 94.153.212.78 15895 Kyivstar PJSC LF_SSHD\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcentminmod%2Fcentminmod-csf-lfd-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcentminmod%2Fcentminmod-csf-lfd-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcentminmod%2Fcentminmod-csf-lfd-parser/lists"}