{"id":50549106,"url":"https://github.com/madgodinc/mgi-pulse","last_synced_at":"2026-06-04T01:04:02.873Z","repository":{"id":361824797,"uuid":"1255979469","full_name":"madgodinc/mgi-pulse","owner":"madgodinc","description":"Terminal log viewer in Rust. 11 formats, regex extraction, native --follow, timeline scrub, query DSL.","archived":false,"fork":false,"pushed_at":"2026-06-03T14:23:38.000Z","size":4820,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-04T01:04:01.824Z","etag":null,"topics":["cli","command-line-tool","journalctl","log-analysis","log-parser","log-viewer","logfmt","logs","ndjson","ratatui","rust","structured-logging","syslog","terminal","tui"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/madgodinc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-01T10:53:23.000Z","updated_at":"2026-06-03T14:36:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/madgodinc/mgi-pulse","commit_stats":null,"previous_names":["madgodinc/mgi-pulse"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/madgodinc/mgi-pulse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madgodinc%2Fmgi-pulse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madgodinc%2Fmgi-pulse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madgodinc%2Fmgi-pulse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madgodinc%2Fmgi-pulse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/madgodinc","download_url":"https://codeload.github.com/madgodinc/mgi-pulse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/madgodinc%2Fmgi-pulse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33886159,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-03T02:00:06.370Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","command-line-tool","journalctl","log-analysis","log-parser","log-viewer","logfmt","logs","ndjson","ratatui","rust","structured-logging","syslog","terminal","tui"],"created_at":"2026-06-04T01:04:02.013Z","updated_at":"2026-06-04T01:04:02.866Z","avatar_url":"https://github.com/madgodinc.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mgi-pulse\n\nA TUI navigator for local log files. NDJSON-first, ten more formats, regex\nescape hatch for the rest.\n\n![mgi-pulse on a 2 GB / 11 M-record NDJSON fixture: a stacked-band timeline reads an incident from across 25 minutes of logs, severity tabs with the framed active tab on the left, typed auto-columns](docs/screenshots/01-hero-ndjson.png)\n\n## Why\n\nI tail `tail -F` and grep `journalctl` like everyone else, and most of the\ntime that's fine. The case where it isn't: you've got a 2 GB structured log\non disk, you want to see when errors clustered, drill into a request id,\nhop to the surrounding context, and you don't want to spin up Loki / Vector\n/ a docker-compose to do it once.\n\nExisting tools sit at two extremes. On one side `less` and `tail -F` —\nfast, ubiquitous, treat everything as text. On the other a whole pipeline\n— a daemon, a schema, a UI. There's a gap in the middle for \"one file, one\nsession, structured awareness, no setup\". `lnav` covers part of it and\n[`toolong`](https://github.com/Textualize/toolong) used to before it was\ndeprecated. `mgi-pulse` is the same niche written in Rust with a few more\nformats and a different keyboard model.\n\nWhat it does that text tools don't:\n\n- Parses each record (NDJSON, logfmt, syslog, …) and offers a typed table\n  instead of a wall of text.\n- Indexes the file once, then filters and timeline-navigates against the\n  in-memory indexes — no daemon, no on-disk index, no config file.\n- Falls back to a `less`-style view when the input has no structure to\n  navigate, so it's safe to point at `/var/log/messages` blind.\n\n## Install\n\nPre-built Linux x86_64 binary, musl-static, no runtime deps:\n\n```sh\ncurl -L https://github.com/madgodinc/mgi-pulse/releases/latest/download/mgi-pulse-x86_64-unknown-linux-musl.tar.gz | tar -xz\n./mgi-pulse --help\n```\n\nCargo from source (Rust ≥ 1.83):\n\n```sh\ncargo install --git https://github.com/madgodinc/mgi-pulse mgi-pulse\n```\n\nmacOS and Windows builds are CI-checked but not released as binaries —\nbuild from source on those platforms.\n\n## Quickstart\n\n```sh\n# One file, autodetect format. NDJSON, logfmt, EDN, syslog, journalctl,\n# logback, Apache/nginx access, and JSON-arrays all auto-detect.\nmgi-pulse app.log\n\n# Force a format when autodetect would misfire (CSV/TSV/Python/regex\n# don't autodetect reliably).\nmgi-pulse --format=python  app.log\nmgi-pulse --format=csv     audit.csv\nmgi-pulse --format=syslog  /var/log/syslog\n\n# Custom log format via named-capture regex. Any plain-text log.\nmgi-pulse --pattern='(?P\u003cts\u003e\\d{4}-\\d{2}-\\d{2}\\s\\d{2}:\\d{2}:\\d{2})\\s+(?P\u003clevel\u003e\\w+)\\s+(?P\u003cmsg\u003e.*)' \\\n          /opt/myapp/log.txt\n\n# Multiple files: merged by timestamp into one time-sorted stream.\nmgi-pulse a.ndjson b.ndjson c.ndjson\n\n# Pipe in stdin. Works for live tails.\ntail -F live.log | mgi-pulse -\n\n# Or native follow — backfill the existing file, then a background thread\n# follows new appends. Handles logrotate's create/rename and copytruncate.\nmgi-pulse --follow live.log\n\n# Compressed input. gzip and zstd are detected by magic bytes, not\n# extension.\nmgi-pulse app.log.gz\nmgi-pulse archive.log.zst\n```\n\nInside the TUI: `Tab` cycles severity views, `/` opens regex search, `:`\nopens the query DSL, `f` opens a `field=value` filter, `t` jumps to a\ntimestamp, `?` shows the stats overlay, `s` saves the filtered view to a\nfile, `q` quits. Full keyboard reference below.\n\n![Split view: the table on the left stays scrollable while the detail pane on the right shows the focused record with every field pretty-printed (level, logger, msg, payload, request_id, ts)](docs/screenshots/02-detail-pane.png)\n\n## Supported inputs\n\n| Format | Flag | Auto-detected | Notes |\n|---|---|---|---|\n| NDJSON | `--format=ndjson` | yes | The default fallback. `--time-field`/`--level-field` override non-standard names. |\n| logfmt | `--format=logfmt` | yes | Go / Heroku `key=value key=\"quoted\"`. |\n| EDN | `--format=edn` | yes | Clojure `{:key value}` maps, `#inst`/`#uuid` tagged literals. |\n| Python `logging` | `--format=python` | no | `YYYY-MM-DD HH:MM:SS,mmm - logger - LEVEL - msg`. Tracebacks fold. |\n| Java logback / log4j2 | `--format=logback` (`log4j`, `java`) | yes | Spring Boot default pattern. Stack traces fold. |\n| Syslog RFC 5424 | `--format=syslog` | yes | `\u003cPRI\u003eVERSION TS HOST APP PID MSGID SD MSG`. Structured-data exposed as `SD-ID.key` fields. |\n| systemd journalctl JSON | `--format=journalctl` (`journal`, `systemd`) | yes | `journalctl -o json`. PRIORITY maps to severity, `__REALTIME_TIMESTAMP` to time. |\n| Apache / nginx access | `--format=access` | yes | Common + Combined Log Format. Severity from HTTP status. |\n| CSV / TSV | `--format=csv` / `tsv` | no | RFC 4180 quoting. First row is the column header. |\n| JSON array of objects | (auto) | yes | `[{...},{...}]` files are flattened to NDJSON. 64 MB cap; bigger → `jq -c '.[]'`. |\n| Generic regex | `--pattern='...'` | no | Named captures `ts`, `level`, plus any other group, become fields. |\n| Anything else | (none) | — | Falls into less-mode: line numbers, raw payload, regex search still works. |\n\nPlus gzip and zstd decompression on any of the above (detected by magic\nbytes, not extension).\n\n## Demo\n\nThe hero screenshot above shows an NDJSON fixture with 11 M records over\n25 minutes. The timeline at the top is four severity bands stacked — top\nis `Error+Fatal`, then `Warn`, `Info`, `Debug+Trace`. The bursts of red\nin the timeline are the incident; the table below is the matching records.\n\nThe split below it (`d` toggles) shows what a record's payload looks like\nexpanded:\n\n![](docs/screenshots/02-detail-pane.png)\n\nSeverity tabs (`Tab` cycles, `1`-`4` quick-jumps):\n\n![Severity tab filtering on Warn — same dataset, only the Warn band stays lit in the timeline; framed active tab makes the current view unambiguous at a glance](docs/screenshots/06-severity-tab.png)\n\nThe DSL prompt (`:` or `;`):\n\n![DSL filter `logger=aurora.tts AND msg~/timeout/` narrows 11 M records to 203 698 matches; the timeline and the table both update to the result set](docs/screenshots/05-dsl-query.png)\n\nLess-mode fallback when the input has no parseable structure:\n\n![Less-mode on a Clojure log4j file — line numbers and raw payload, no empty columns](docs/screenshots/03-less-mode.png)\n\nIn less-mode the detail pane shows ±5 lines of context so stack traces\nread as a block:\n\n![Detail pane in less-mode — ±5 lines of stack-trace context beside the focused row](docs/screenshots/04-less-mode-context.png)\n\n## Query DSL\n\nPress `:` (or `;` if `:` is awkward on your layout — Russian keyboards,\nsome Mac layouts) to enter a one-line expression. It compiles into the\nsame predicate machinery the table filters already use:\n\n```text\nlevel=error\nlevel=error AND msg~/timeout/\n(level=error OR level=warn) AND NOT logger=health-check\nlevel=error AND (msg~/timeout/ OR msg~/refused/)\nts\u003e=2026-06-01T12:00 AND ts\u003c2026-06-01T13:00\nlogger=my.app AND msg~/conn(ection)? lost/ AND level!=debug\n```\n\nOperators on fields: `=`, `!=`, `~/regex/`, and `\u003e`, `\u003e=`, `\u003c`, `\u003c=` (the\ncomparison ops only apply to `ts`). Boolean composition: `AND`, `OR`,\n`NOT`, parentheses. Precedence is the conventional one — `NOT` binds\ntightest, then `AND`, then `OR`. Keywords are uppercase ASCII so they\ndon't shadow field names like `and_count`.\n\nSyntax errors surface in the status bar before any scan runs.\n\n## Keyboard\n\n| Key | Action | Notes |\n|---|---|---|\n| `q` / `Ctrl-C` | Quit | |\n| `/` | Regex search | `Enter` applies, `Esc` cancels |\n| `:` / `;` | Query DSL prompt | `;` is the alt-binding for layouts where `:` needs a modifier |\n| `f` | `field=value` filter | Composes with regex and DSL via AND |\n| `t` | Jump to a timestamp | RFC3339 prefix, e.g. `2026-06-01T12:00` |\n| `s` | Save filtered view to a file | One record per line |\n| `?` | Stats overlay | Per-severity counts, time span, top values |\n| `]` / `[` | Widen / narrow the auto-columns cap | No restart needed |\n| `\u003c` / `\u003e` | Move the timeline scrub cursor | First press activates scrub; Shift jumps 10 |\n| `+` / `-` | Zoom timeline | Halve / double, anchored on the cursor |\n| `Enter` | Apply scrub as a time-range filter | Single bin if no zoom, the full window otherwise |\n| `d` | Detail pane | Pretty-printed JSON; ±5-line context in less-mode |\n| `m` | Severity strict / min-mode | `Warn` vs `Warn+` |\n| `0`–`4` | Severity quick-filter | `0` clears, `1` Error+Fatal, `2` Warn, `3` Info, `4` Debug+Trace |\n| `b` / `B` | Toggle / jump-to-next bookmark | Yellow ★ in the gutter |\n| `Esc` | Cancel scrub, or clear filters | First press cancels a scrub; second clears filters |\n| `R` | Rescan schema | Useful when the first 10k records were a boot banner |\n| `Tab` / `Shift-Tab` | Next / previous tab | |\n| `Ctrl-T` / `Ctrl-W` | New / close tab | Last close quits |\n| `Up`/`Down`, `PgUp`/`PgDn`, `g`/`G` | Cursor movement | One row, 20 rows, start/end |\n\n### Mouse and terminal selection\n\nMouse capture is on by default: the wheel scrolls the table, clicks select\ntabs. That intercepts the terminal's own text selection — hold `Shift`\nwhile you drag to let the terminal handle the selection directly. Works\nin WezTerm, Alacritty, GNOME-Terminal, Konsole, iTerm2. Pass `--no-mouse`\nto disable capture entirely if you'd rather keep unmodified terminal\nselection (useful over SSH or with a copy-on-select setup).\n\n## Custom field names\n\nNDJSON variants differ on which key holds the timestamp and level. The\ndefaults are `ts` and `level`; override with `--time-field` and\n`--level-field`:\n\n```sh\nmgi-pulse --time-field=@timestamp --level-field=severity_text app.log   # ECS\nmgi-pulse --time-field=@t app.log                                       # Serilog\nmgi-pulse --time-field=eventTime app.log                                # k8s audit\n```\n\n## Following live files\n\nThree ways, each safe for a different scenario:\n\n```sh\n# 1. Native follow. Backfills the existing content, then a background\n#    thread reads appends through a crossbeam channel. Handles\n#    logrotate's create/rename mode (inode change) and copytruncate\n#    (file shrinks below the read cursor).\nmgi-pulse --follow live.log\n\n# 2. Pipe through tail -F. The most portable option, because tail -F\n#    is older than most of us and has debugged every rotation edge\n#    case across every Unix.\ntail -F live.log | mgi-pulse -\n\n# 3. journalctl, kubectl logs, docker logs — same shape, stdin.\njournalctl -u myapp -f -o json | mgi-pulse --format=journalctl -\n```\n\nA note on `mmap` and live files: `mgi-pulse` mmaps plain files for the\nfast indexing path. A file that's truncated by another process while\nyou're viewing it will deliver SIGBUS — Unix mechanic, not a Rust error,\nnot catchable. So **don't open an actively-rotating file with `mgi-pulse\napp.log` directly**; use one of the three options above. Static log\nsnapshots (`app.log.1`, archived dates) are fine.\n\n## Bookmarks persistence\n\n`b` toggles a bookmark on the focused row, `B` cycles through them. For\nsingle-file sources, bookmarks survive between sessions via a JSON\nsidecar at `$XDG_DATA_HOME/mgi-pulse/bookmarks.json` (default\n`~/.local/share/mgi-pulse/bookmarks.json`). The sidecar is keyed by the\nfile's inode and size; a rotated or truncated file drops its saved\nbookmarks automatically so the marks don't land on unrelated content.\nStdin and merged sources skip persistence — no stable identity to\nkey on.\n\n## What it doesn't do\n\n- **Plain-text without `--pattern`.** Unstructured logs fall into\n  less-mode — usable, but no typed fields, no severity tabs, no\n  per-field DSL. Pass `--pattern='(?P\u003cts\u003e...)...'` to lift them into\n  the structured path.\n- **Files larger than RAM minus index headroom.** The index lives\n  in memory (one entry per record across three parallel arrays); a\n  30 GB file is fine on a 64 GB box but blocks the UI during the\n  synchronous backfill.\n- **Live editing of running queries with very large indexes.**\n  Predicates re-scan the index on each filter change. Sub-second on\n  2 GB / 11 M records (i5-12400F); minutes on 100 GB.\n- **Remote / multi-host / persistence between machines.** There's no\n  daemon and no on-disk index. If you need to correlate across a\n  fleet, you want Loki, Vector, an OTEL collector, not this.\n- **Windows-native rotation handling.** The follow worker's inode\n  check falls back to file size on non-Unix platforms; copytruncate\n  is detected the same way (size shrink) but rename-rotation on\n  Windows may not be picked up until the file shrinks or grows.\n\n## Comparison\n\nRoughly the same niche:\n\n- **`lnav`** is the prior art and the closest peer — battle-tested,\n  C++, broader format library, SQL queries. `mgi-pulse` is younger,\n  fewer formats today, but a different keyboard model (severity tabs,\n  one-line DSL instead of SQL) and a Rust toolchain if that matters\n  to you.\n- **`toolong`** had the same idea in Python on top of Textual; the\n  Textualize team [archived it](https://github.com/Textualize/toolong)\n  in 2025. `mgi-pulse` covers the same demo workflow.\n- **`klogg`** is the GUI option for big files. If you want a desktop\n  GUI you want `klogg`. If you want a terminal session you want this\n  or `lnav`.\n\nNot the same niche:\n\n- **`less`, `tail`, `bat`** — text-first. Use them for \"I just want\n  to see the file\".\n- **Loki / Vector / OTEL collectors** — pipeline-first. Use them for\n  \"I'm running a service and want to see across instances\".\n\n## Performance reference\n\nEnd-to-end index of a 2 GB / 11 M-record synthetic NDJSON file:\n**~2.8 seconds** on an i5-12400F (6c/12t, ext4 on NVMe). Throughput\n~700 MB/s, ~4 M records/s. See [BENCHMARKS.md](BENCHMARKS.md) for the\nhot-path breakdown and what's explicitly not measured.\n\nThese are reference numbers from one dev box, not a guarantee. Real\nworkloads on real boxes will vary; the indexer-bench binary in\n`bench/parse-bench/` reproduces them for your hardware.\n\n## License\n\nApache-2.0. See [LICENSE](LICENSE).\n\n## Acknowledgments\n\nBuilt on:\n\n- [ratatui](https://github.com/ratatui/ratatui) — TUI rendering\n- [crossterm](https://github.com/crossterm-rs/crossterm) — input / mouse\n- [memmap2](https://github.com/RazrFalcon/memmap2-rs) — mmap\n- [serde](https://serde.rs/) / [serde_json](https://github.com/serde-rs/json) — borrowed JSON parsing\n- [regex](https://github.com/rust-lang/regex) — search and `--pattern`\n- [flate2](https://github.com/rust-lang/flate2-rs) / [zstd](https://github.com/gyscos/zstd-rs) — decompression\n- [crossbeam-channel](https://github.com/crossbeam-rs/crossbeam) — follow-worker → UI\n\nInspired by [`lnav`](https://github.com/tstack/lnav) and\n[`toolong`](https://github.com/Textualize/toolong) — same idea, different\ntradeoffs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadgodinc%2Fmgi-pulse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmadgodinc%2Fmgi-pulse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmadgodinc%2Fmgi-pulse/lists"}