{"id":18351123,"url":"https://github.com/simagix/hatchet","last_synced_at":"2025-05-16T11:05:54.108Z","repository":{"id":65277111,"uuid":"582474370","full_name":"simagix/hatchet","owner":"simagix","description":"MongoDB JSON Log Analyzer","archived":false,"fork":false,"pushed_at":"2025-01-30T21:58:16.000Z","size":81502,"stargazers_count":198,"open_issues_count":8,"forks_count":25,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-04-12T08:17:28.374Z","etag":null,"topics":["golang","json","mongodb","sqlite3"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simagix.png","metadata":{"files":{"readme":"README.md","changelog":"change_logs","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"audit_template.go","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-12-27T00:29:17.000Z","updated_at":"2025-04-02T14:23:05.000Z","dependencies_parsed_at":"2024-04-19T02:29:24.212Z","dependency_job_id":"2ae5fcbc-4905-4064-8079-e9a40abd2157","html_url":"https://github.com/simagix/hatchet","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simagix%2Fhatchet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simagix%2Fhatchet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simagix%2Fhatchet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simagix%2Fhatchet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simagix","download_url":"https://codeload.github.com/simagix/hatchet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254518383,"owners_count":22084374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","json","mongodb","sqlite3"],"created_at":"2024-11-05T21:29:35.665Z","updated_at":"2025-05-16T11:05:49.096Z","avatar_url":"https://github.com/simagix.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hatchet - MongoDB JSON Log Analyzer and Viewer\nHatchet is a powerful and sophisticated logs analyzer and viewer specifically designed for MongoDB JSON logs. It provides advanced features for logs processing, aggregation and storage of the processed data. To make the data accessible and usable for its users, Hatchet utilizes an embedded SQLite3 database. This database allows for the storage of processed and aggregated data and makes it possible to offer RESTful APIs and a web interface to users.\n\nThe web interface of Hatchet is highly interactive and user-friendly, providing a seamless experience for searching logs and navigating through reports and charts. The intuitive design and easy-to-use interface makes it simple for users to find what they need, when they need it. Additionally, with the embedded database, Hatchet provides fast access to data and a high level of performance, making it the ideal solution for logs analysis and management. Further design details can be found at [![Hatchet: Empowering Smart MongoDB Log Analysis](http://img.youtube.com/vi/WavOyaFTDE8/0.jpg)](https://www.youtube.com/watch?v=WavOyaFTDE8).\n\n## Change Log\n- [v0.5.0](https://youtu.be/4RkeMOOAtv8), August, 2023\n\n## Build\nClone and run the *build.sh* script; *gcc* is required to support CGO.\n```bash\ngit clone --depth 1 https://github.com/simagix/hatchet.git\ncd hatchet ; ./build.sh\n```\n\nAn executable *hatchet* is output to the directory *dist/*.  Note that the script also works and tested on Windows x64 using MingGW and Git Bash.\n\n## Quick Start\nUse the command below to process a log file, mongod.log.gz and start a web server listening to port 3721.  The default database is SQLite3.\n```bash\n./dist/hatchet -web logs/sample-mongod.log.gz\n```\n\nLoad a file within a defined time range:\n```bash\n./dist/hatchet -web -from \"2023-09-23T20:25:00\" -to \"2023-09-23T20:26:00\" logs/sample-mongod.log.gz\n```\n\nLoad multiple files and process them individually:\n```bash\n./dist/hatchet -web rs1/mongod.log rs2/mongod.log rs3/mongod.log\n```\n\nLoad multiple files and process them collectively:\n```bash\n./dist/hatchet -web -merge rs1/mongod.log rs2/mongod.log rs3/mongod.log\n```\n\nUse the URL `http://localhost:3721/` in a browser to view reports and charts.  Alternatively, you can use the *in-memory* mode without persisting data, for example:\n```bash\n./dist/hatchet -url in-memory logs/sample-mongod.log.gz\n```\n\nif you choose to view in the legacy format without a browser, use the command below:\n```bash\n./dist/hatchet -legacy logs/sample-mongod.log.gz\n```\n\nFor additional usages and integration details, see [developer's guide](README_DEV.md).\n\n## A Smart Log Analyzer\nHow smart Hatchet is?  A picture is worth a thousand words.\n\n![Sage Says](sage_says.png)\n\n## Other Usages\nOther than its ability to read from files, Hatchet offers additional functionality that includes reading from S3 and web servers, as well as MongoDB Atlas. This means that users can use Hatchet to conveniently access and download data from these sources, providing a more versatile and efficient data analysis experience.\n\n### Web Servers\nThe tool supports reading from web servers using both the *http://* and *https://* protocols. The `-user` flag is optional when using basic authentication.\n\n```bash\nhatchet [-user {username}:{password}] https://{hostname}/{log name}\n```\n\n### Atlas\nTo download logs directly from MongoDB Atlas, you will need to use the `-user` and `-digest` flags and provide the necessary information for both. These flags are used to authenticate and authorize your access to the database.\n\n```bash\nhatchet -user {pub key}:{private key} -digest https://cloud.mongodb.com/api/atlas/v1.0/groups/{group ID}/clusters/{hostname}/logs/mongodb.gz\n```\n\n### AWS S3\nHatchet has the ability to download files from AWS S3. When downloading files, Hatchet will automatically retrieve the *Region* and *Credentials* information from the configuration files located at *${HOME}/.aws*. This means that there's no need to provide this information manually each time you download files from AWS S3 using Hatchet.\n\n```bash\nhatchet -s3 [--endpoint-url {test endpoint}] {bucket}/{key name}\n```\n\n## Logs Obfuscation\nUse Hatchet to obfuscate logs. It automatically obfuscates the values of the matched patterns under the \"attr\" field, such as SSN, credit card numbers, phone numbers, email addresses, IP addresses, FQDNs, port numbers, namespaces, and other numbers. Note that, for example, replacing \"host.example.com\" with \"rose.taipei.com\" in the log file will consistently replace all other occurrences of \"host.example.com\" with \"rose.taipei.com\". To obfuscate logs and redirect them to a file, use the following syntax:\n\n```bash\nhatchet -obfuscate {log file} \u003e {output file}\n```\n\n## License\n[Apache-2.0 License](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimagix%2Fhatchet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimagix%2Fhatchet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimagix%2Fhatchet/lists"}