{"id":26756937,"url":"https://github.com/donaldc24/timberjack","last_synced_at":"2025-04-15T03:52:06.695Z","repository":{"id":283884175,"uuid":"953172554","full_name":"donaldc24/timberjack","owner":"donaldc24","description":"A fast CLI tool built in Rust that analyzes log files, extracts patterns, and generates insights.","archived":false,"fork":false,"pushed_at":"2025-04-14T18:44:17.000Z","size":282,"stargazers_count":5,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T03:52:01.687Z","etag":null,"topics":["cli","command-line","debugging","developer-tools","log-analysis","log-parser","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/donaldc24.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-22T18:26:42.000Z","updated_at":"2025-04-04T17:54:03.000Z","dependencies_parsed_at":"2025-03-22T20:34:54.849Z","dependency_job_id":null,"html_url":"https://github.com/donaldc24/timberjack","commit_stats":null,"previous_names":["donaldc24/timber","donaldc24/timberjack"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donaldc24%2Ftimberjack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donaldc24%2Ftimberjack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donaldc24%2Ftimberjack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/donaldc24%2Ftimberjack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/donaldc24","download_url":"https://codeload.github.com/donaldc24/timberjack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249003954,"owners_count":21196794,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","command-line","debugging","developer-tools","log-analysis","log-parser","rust"],"created_at":"2025-03-28T15:22:00.038Z","updated_at":"2025-04-15T03:52:06.689Z","avatar_url":"https://github.com/donaldc24.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Timberjack 🪓\n\n[![Rust CI](https://github.com/donaldc24/timberjack/workflows/Rust%20CI/badge.svg)](https://github.com/donaldc24/timberjack/actions)\n[![Crates.io](https://img.shields.io/crates/v/timberjack.svg)](https://crates.io/crates/timberjack)\n[![Security Audit](https://github.com/donaldc24/timberjack/workflows/Security%20audit/badge.svg)](https://github.com/donaldc24/timberjack/actions?query=workflow%3A%22Security+audit%22)\n\n**Timberjack: Fell Your Logs Fast** - A lightning-fast CLI log analysis tool built in Rust.\n\n## 📋 Overview\n\nTimberjack is a log-agnostic CLI tool that chops through noise to deliver patterns, trends, and stats from your logs. It's designed to be portable, requiring no servers or complex setup, and works with logs from any source—Java, Rust, Python, or any text-based logs.\n\n## ✨ Key Features\n\n- **Fast JSON Processing**: 70-90x faster than jq for large JSON logs\n- **Pattern Matching**: On par with grep for small files, with richer analysis capabilities\n- **Parallel Processing**: Up to 57% faster analysis of large files with automatic multi-threading\n- **Statistical Analysis**: Get insights on log levels, error types, and message uniqueness\n- **Memory Efficiency**: Low memory footprint even with large log files\n- **Automatic Format Detection**: Intelligently handles both plaintext and JSON logs\n- **Stdin Support**: Seamless piping and streaming log analysis\n\n## 🚀 Installation\n\n```bash\ncargo install timberjack\n```\n\n## 🌐 Stdin and Piping Superpowers\n\nTimberjack now supports stdin input, making log analysis incredibly flexible:\n\n```bash\n# Pipe logs from any source\ncat system.log | timber --level ERROR\ndocker logs mycontainer | timber --chop \"connection\"\njournalctl | timber --stats\n\n# Quick counting\ncat large_log.log | timber --count\ncat access.log | grep \"404\" | timber --count\n\n# Advanced analysis\nkubectl logs pod/web-server | timber --json --trend\n```\n\n**Pro Tips:**\n- Automatically detects log formats from stdin\n- Supports compressed logs via zcat/gunzip\n- Memory-mapped processing for large input streams\n- Works with JSON, plaintext, and mixed log formats\n\n## 🔨 Quick Examples\n\n```bash\n# Basic log analysis with automatic format detection\ntimber path/to/logfile.log\n\n# Find errors and show statistics\ntimber --level ERROR --stats app.log\n\n# Analyze JSON logs and filter by specific fields\ntimber --format json -f service=api -f status=500 logs.json\n\n# Get just the count of errors (fastest mode)\ntimber --count --level ERROR app.log\n\n# Use parallel processing for large files\ntimber --parallel large_logfile.log\n```\n\n## 📊 Benchmark Results\n\nOur latest benchmarks show impressive performance across various operations:\n\n### JSON Processing (vs jq)\n| File Size | Timberjack | jq | Speedup |\n|-----------|------------|-----|---------|\n| 10K lines | 0.074s | 0.814s | **11x faster** |\n| 100K lines | 0.156s | 7.149s | **46x faster** |\n| 1M lines | 0.967s | 69.529s | **72x faster** |\n\n### Pattern Matching\n| File Size | timber-chop-count | grep | ripgrep |\n|-----------|------------------|------|---------|\n| 10K lines | 0.069s | 0.072s | 0.138s |\n| 100K lines | 0.092s | 0.079s | 0.112s |\n| 1M lines | 0.405s | 0.128s | 0.140s |\n\n### Parallel Processing (10M lines)\n| Operation | Sequential | Parallel | Improvement |\n|-----------|------------|----------|-------------|\n| Standard Processing | 10.609s | 7.157s | **32% faster** |\n| Pattern Matching | 6.541s | 3.514s | **46% faster** |\n| JSON Processing | 30.603s | 12.976s | **58% faster** |\n\n## 📚 Detailed Usage\n\n### Pattern Searching\n\n```bash\n# Find logs containing \"Exception\"\ntimber --chop \"Exception\" app.log\n\n# Count occurrences (fastest)\ntimber --count --chop \"Exception\" app.log\n\n# Combine with level filtering\ntimber --chop \"timeout\" --level ERROR app.log\n```\n\n### JSON Log Analysis\n\n```bash\n# Process JSON logs (auto-detected)\ntimber app.json\n\n# Filter by JSON fields\ntimber --format json -f service=api app.json\n\n# Complex field filtering\ntimber --format json -f service=api -f level=ERROR -f \"response_time\u003e1000\" app.json\n```\n\n### Statistical Analysis\n\n```bash\n# Get comprehensive statistics\ntimber --stats app.log\n\n# Show time-based trends\ntimber --trend app.log\n\n# Show unique messages in stats\ntimber --stats --show-unique app.log\n\n# Output top 10 error types\ntimber --stats --top-errors 10 app.log\n```\n\n### Performance Options\n\n```bash\n# Force parallel processing for large files\ntimber --parallel large.log\n\n# Memory-efficient mode\ntimber --count large.log\n\n# Output JSON for programmatic use\ntimber --stats --json app.log \u003e analysis.json\n```\n\n## 🔍 When to Use Timberjack\n\n- **Complex JSON Logs**: Timberjack outperforms specialized tools like jq by 46-72x on large files\n- **All-in-One Analysis**: Replaces grep, jq, and custom scripts with one unified tool\n- **Large Log Files**: Automatic parallelization for multi-gigabyte logs\n- **Statistical Insights**: When you need more than just matching lines\n- **CI/CD Pipelines**: Fast and reliable log analysis in automated environments\n\n## 🛠️ Command-Line Options\n\n| Option | Description |\n|--------|-------------|\n| `--chop \u003cPATTERN\u003e` | Search for logs matching pattern (regex supported) |\n| `--level \u003cLEVEL\u003e` | Filter by log level (ERROR, WARN, INFO, etc.) |\n| `--trend` | Show time-based trends of log occurrences |\n| `--stats` | Show summary statistics |\n| `--count` | Output only the count (faster) |\n| `--format \u003cFORMAT\u003e` | Specify log format (auto, json) |\n| `-f, --field \u003cFIELD=VALUE\u003e` | Filter by field value (for JSON logs) |\n| `--parallel` | Force parallel processing |\n| `--json` | Output results in JSON format |\n| `--top-errors \u003cN\u003e` | Number of top error types to show (default: 5) |\n| `--show-unique` | Show unique messages in stats output |\n\n## 📝 Roadmap\n\n- VS Code extension (coming May 2025)\n- Multi-file analysis\n- Interactive TUI mode\n- Advanced pattern correlation\n\n## 🤝 Contributing\n\nContributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.\n\n## 📄 License\n\nTimberjack is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonaldc24%2Ftimberjack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdonaldc24%2Ftimberjack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdonaldc24%2Ftimberjack/lists"}