{"id":23674568,"url":"https://github.com/mauricelambert/querycombinedlogformat","last_synced_at":"2026-04-26T20:32:23.458Z","repository":{"id":264650026,"uuid":"893974955","full_name":"mauricelambert/QueryCombinedLogFormat","owner":"mauricelambert","description":"This tool extracts, filters and parses combined log format (apache and nginx default access.log format) with a easy and fast language syntax.","archived":false,"fork":false,"pushed_at":"2024-12-04T19:22:09.000Z","size":77,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-29T13:27:20.661Z","etag":null,"topics":["access","access-log","access-logs","apache","clf","combined-log-format","dfir","filtering","forensic","forensics","incident-response","investigation","language","log","nginx","parser","query","regex","web"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mauricelambert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-25T14:32:33.000Z","updated_at":"2024-12-04T19:26:51.000Z","dependencies_parsed_at":"2024-12-04T20:37:34.626Z","dependency_job_id":null,"html_url":"https://github.com/mauricelambert/QueryCombinedLogFormat","commit_stats":null,"previous_names":["mauricelambert/apacheaccesslogparser","mauricelambert/queryapacheaccesslog","mauricelambert/querycombinedlogformat"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mauricelambert%2FQueryCombinedLogFormat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mauricelambert%2FQueryCombinedLogFormat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mauricelambert%2FQueryCombinedLogFormat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mauricelambert%2FQueryCombinedLogFormat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mauricelambert","download_url":"https://codeload.github.com/mauricelambert/QueryCombinedLogFormat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239707043,"owners_count":19683990,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["access","access-log","access-logs","apache","clf","combined-log-format","dfir","filtering","forensic","forensics","incident-response","investigation","language","log","nginx","parser","query","regex","web"],"created_at":"2024-12-29T13:27:36.458Z","updated_at":"2025-12-23T13:30:22.511Z","avatar_url":"https://github.com/mauricelambert.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![QueryCombinedLogFormat Logo](https://mauricelambert.github.io/info/python/security/QueryCombinedLogFormat_small.png \"QueryCombinedLogFormat logo\")\r\n\r\n# QueryCombinedLogFormat\r\n\r\n## Description\r\n\r\nThis tool extracts, filters and parses combined log format (apache and nginx default access.log format) with a easy and fast language syntax.\r\n\r\n\u003e This tool has been written in forensic lessons and challenges for certification. It's a little tool to reduce time for analysis.\r\n\u003e\u003e - This tool implements a basic and permissive syntax to query combined log files (apache and nginx default access.log format) with details and typing.\r\n\u003e\u003e - This tool can parses multiples logs files using glob syntax and parses Gzip compressed logs files. If you have configured the logs rotation and gzip compression you can use this tool to analyze all of your logs with a simple command line.\r\n\u003e\u003e - With this tool you can extract logs in a CSV and mJSON format to analyse it faster when you start this script multiple times on the same logs (in incident response). You can use the CSV file in excel with filtered logs for analyze or retex.\r\n\u003e\u003e - To identify faster suspicious logs, this script implements a statistics option to make a CLI table with values and counters.\r\n\r\n## Requirements\r\n\r\nThis package require:\r\n - python3\r\n - python3 Standard Library\r\n\r\n## Installation\r\n\r\n### Pip\r\n\r\n```bash\r\npython3 -m pip install QueryCombinedLogFormat\r\n```\r\n\r\n### Git\r\n\r\n```bash\r\ngit clone \"https://github.com/mauricelambert/QueryCombinedLogFormat.git\"\r\ncd \"QueryCombinedLogFormat\"\r\npython3 -m pip install .\r\n```\r\n\r\n### Wget\r\n\r\n```bash\r\nwget https://github.com/mauricelambert/QueryCombinedLogFormat/archive/refs/heads/main.zip\r\nunzip main.zip\r\ncd QueryCombinedLogFormat-main\r\npython3 -m pip install .\r\n```\r\n\r\n### cURL\r\n\r\n```bash\r\ncurl -O https://github.com/mauricelambert/QueryCombinedLogFormat/archive/refs/heads/main.zip\r\nunzip main.zip\r\ncd QueryCombinedLogFormat-main\r\npython3 -m pip install .\r\n```\r\n\r\n## Usages\r\n\r\n### Command line\r\n\r\n```bash\r\nQueryCombinedLogFormat              # Using CLI package executable\r\npython3 -m QueryCombinedLogFormat   # Using python module\r\npython3 QueryCombinedLogFormat.pyz  # Using python executable\r\nQueryCombinedLogFormat.exe          # Using python Windows executable\r\n\r\nQueryCombinedLogFormat [-s|--statistics] [-d|--to-db] \u003cglob_syntax_log_files\u003e \u003cqueries\u003e...\r\n\r\nQueryCombinedLogFormat -d 'access.log*' \"method = POST\" 'status ~ 5??' # print logs and generate a DB file with POST method or server error (http status 5XX)\r\nQueryCombinedLogFormat -s 'access_log_db_*.csv' '(METHOD = post or url ~ *admin*) \u0026 (ip \u003e 91.0.0.0 | referrer ~ *://*)' # use the precedent generated DB to get statistics for POST request or admin URL for all IP address greater than 91.0.0.0 or with a url referrer\r\n```\r\n\r\n### Query syntax\r\n\r\n#### Examples\r\n\r\n1. Query all requests with the *method* POST: `method = POST`\r\n2. Query all requests with a *status code* starting by 5 (server error): `status ~ 5??`\r\n3. Query all requests with response size greater or equal than 60000000: `size \u003e= 60000000`\r\n4. Query all requests with a specific match on *User-Agent* and a specific IP address: `user_agent ~ *Version/6.0\\ Mobile* and ip = 66.249.73.135`\r\n5. Query all requests with the *method* POST or `admin` in URL if IP address is greater than `91.0.0.0` and referrer is not empty (contains URL instead of `-`): `(METHOD = post or url ~ *admin*) \u0026 (ip \u003e 91.0.0.0 | referrer ~ *://*)`\r\n\r\n### Fields\r\n\r\n1. `ip` (IPv4Address)\r\n2. `datetime` (datetime)\r\n3. `method` (string)\r\n4. `url` (string)\r\n5. `version` (float)\r\n6. `status` (int)\r\n7. `size` (int)\r\n8. `referrer` (string)\r\n9. `user_agent` (string)\r\n\r\n### Operators\r\n\r\n1. `=`\r\n2. `~`\r\n3. `\u003e`\r\n4. `\u003c`\r\n5. `\u003e=`\r\n6. `\u003c=`\r\n7. `!`\r\n\r\n### Inter expression\r\n\r\n1. `and`\r\n2. `\u0026`\r\n3. `or`\r\n4. `|`\r\n\r\n### Priority\r\n\r\n1. Parenthesis\r\n2. Left to right\r\n\r\n### Escape character\r\n\r\n`\\` works only before a *spaces* or *operators* characters else is the `\\` character.\r\n\r\n## Links\r\n\r\n - [Pypi](https://pypi.org/project/QueryCombinedLogFormat)\r\n - [Github](https://github.com/mauricelambert/QueryCombinedLogFormat)\r\n - [Documentation](https://mauricelambert.github.io/info/python/security/QueryCombinedLogFormat.html)\r\n - [Python executable](https://mauricelambert.github.io/info/python/security/QueryCombinedLogFormat.pyz)\r\n - [Python Windows executable](https://mauricelambert.github.io/info/python/security/QueryCombinedLogFormat.exe)\r\n\r\n## License\r\n\r\nLicensed under the [GPL, version 3](https://www.gnu.org/licenses/).\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmauricelambert%2Fquerycombinedlogformat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmauricelambert%2Fquerycombinedlogformat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmauricelambert%2Fquerycombinedlogformat/lists"}