{"id":20010196,"url":"https://github.com/activecm/zcutter","last_synced_at":"2025-05-04T20:30:44.416Z","repository":{"id":156966514,"uuid":"629554703","full_name":"activecm/zcutter","owner":"activecm","description":"Extracts fields from zeek logs, compatible with zeek-cut","archived":false,"fork":false,"pushed_at":"2024-07-10T18:25:34.000Z","size":151,"stargazers_count":21,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-30T22:42:00.476Z","etag":null,"topics":["converter","json","python-3","python3","tsv","tsv-parser","zeek"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/activecm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-18T14:46:02.000Z","updated_at":"2025-03-17T00:57:28.000Z","dependencies_parsed_at":"2024-02-13T20:49:09.439Z","dependency_job_id":null,"html_url":"https://github.com/activecm/zcutter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activecm%2Fzcutter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activecm%2Fzcutter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activecm%2Fzcutter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/activecm%2Fzcutter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/activecm","download_url":"https://codeload.github.com/activecm/zcutter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252395174,"owners_count":21740977,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["converter","json","python-3","python3","tsv","tsv-parser","zeek"],"created_at":"2024-11-13T07:18:48.534Z","updated_at":"2025-05-04T20:30:44.017Z","avatar_url":"https://github.com/activecm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n![zcutter](zcutter_icon_tag.png)\n\n# zcutter, compatible with zeek-cut (formerly bro-cut)\n\nThis is a python script that, like zeek-cut, handles the task of\nextracting specific columns from zeek-logs.  It can also be used to\nconvert between TSV and json format log files.\n\n## Primary differences\n- zcutter is a standalone python script with no dependencies other than\npython3 (which should be on all Linuxes and MacOS.)\n\n- zeek-cut only reads uncompressed TSV delimited zeek logs.  zcutter will\nread and write TSV and json format.\n\n- The \"-r\" command line parameter accepts any number of input files, and\nthese can be any mix of gz compressed, bzip2 compressed, uncompressed,\nTSV, and json.  Compressed files will be automatically decompressed on\nthe fly and the temporary files will be deleted when done.\n\n- zcutter will write out the (converted) files to an output directory,\nallowing you to bulk convert zeek logs.\n\n\n# Quickstart\n```\nmkdir -p ~/bin/\ncd ~/bin/\nwget https://raw.githubusercontent.com/activecm/zcutter/main/zcutter.py -O zcutter.py\nchmod 755 zcutter.py\nif ! type zeek-cut \u003e/dev/null 2\u003e\u00261 ; then ln -s zcutter.py zeek-cut ; fi\n```\n- For Python 3.5, use zcutter-stripped.py .  (Python 3.6 and above use zcutter.py)\n\n\n# Example commands\n\n\n- Look at the source IP, method, host, and URI fields from an http log:\n\n`zcat http.00\\:00\\:00-01\\:00\\:00.log.gz | nice zcutter.py id.orig_h method host uri -C | less -S -x 20`\n\n\n- Same as above, but automatically decompress input logs\n\n`nice zcutter.py id.orig_h method host uri -C -r http.00\\:00\\:00-01\\:00\\:00.log.gz | less -S -x20`\n\n\n- Convert all gzip compressed logs (except conn-summary logs) in this\ndirectory to json and save the uncompressed json logs in ~/json-out/ :\n\n`zcutter.py -j -o ~/json-out/ -r *.log.gz`\n\n\n- Like above, but compress the output logs at the end if successful:\n\n`zcutter.py -j -o ~/json-out/ -r *.log.gz \u0026\u0026 gzip -9 ~/json-out/*.log`\n\n\n- Like above, but preserve the paths under /V/source in /V/dest/ and compress with bzip2.  The file glob after -r needs to match the number of levels down where the .log files are found:\n\n```\ncd /V/source/\nzcutter.py -o /V/dest/ -j -r */*/*.log.gz\nfind /V/dest/ -mmin +1 -iname '*.log' -print0 | xargs -r -n 50 -0 nice -n 19 bzip2 -9\n```\n\n\n\n- To see the command line options:\n\n`zcutter.py -h`\n\nThe current help text:\n\n```\nusage: zcutter.py [-h] [-n] [-c] [-C] [-m] [-M] [-F FIELDSEPARATOR] [-d] [-D DATEFORMAT] [-t] [-j] [-v] [-o OUTPUTDIR] [-r [READ ...]] [fields ...]\n\nzcutter.py version 0.1.8: returns fields from zeek log files.\n\npositional arguments:\n  fields                fields to display\n\noptions:\n  -h, --help            show this help message and exit\n  -n, --negate          Negate test; show all columns EXCEPT those specified.\n  -c, --firstheaders    Include first format header blocks in the output.\n  -C, --allheaders      Include all format header blocks in the output.\n  -m, --firstminheaders\n                        Include first format header blocks in the output in minimal view.\n  -M, --allminheaders   Include all format header blocks in the output in minimal view.\n  -F FIELDSEPARATOR, --fieldseparator FIELDSEPARATOR\n                        character that separates output fields.\n  -d, --readabledate    Conert ts to readable format.\n  -D DATEFORMAT, --dateformat DATEFORMAT\n                        Format to use for date output.\n  -t, --tsv             Force TSV output\n  -j, --json            Force json output\n  -v, --verbose         Be verbose\n  -o OUTPUTDIR, --outputdir OUTPUTDIR\n                        Directory in which to place corresponding (uncompressed) output files\n  -r [READ ...], --read [READ ...]\n                        Log file(s) from which to read logs (place this option last)\n```\n\n\n# References\n\n[zcutter repository](https://github.com/activecm/zcutter/)\n\n[zeek-cut repository](https://github.com/zeek/zeek-aux/)\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Factivecm%2Fzcutter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Factivecm%2Fzcutter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Factivecm%2Fzcutter/lists"}