{"id":35135500,"url":"https://github.com/nh13/snakesee","last_synced_at":"2026-04-02T17:52:03.399Z","repository":{"id":329488838,"uuid":"1117635480","full_name":"nh13/snakesee","owner":"nh13","description":"A terminal UI for monitoring Snakemake workflows","archived":false,"fork":false,"pushed_at":"2026-03-28T05:32:04.000Z","size":841,"stargazers_count":16,"open_issues_count":7,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-28T08:41:09.026Z","etag":null,"topics":["bioinformatics","monitor","python","snakemake","terminal","tui","workflow"],"latest_commit_sha":null,"homepage":"https://snakesee.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nh13.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-16T15:39:39.000Z","updated_at":"2026-03-28T05:32:06.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/nh13/snakesee","commit_stats":null,"previous_names":["nh13/snakesee"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/nh13/snakesee","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nh13%2Fsnakesee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nh13%2Fsnakesee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nh13%2Fsnakesee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nh13%2Fsnakesee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nh13","download_url":"https://codeload.github.com/nh13/snakesee/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nh13%2Fsnakesee/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31312744,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T12:59:32.332Z","status":"ssl_error","status_checked_at":"2026-04-02T12:54:48.875Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","monitor","python","snakemake","terminal","tui","workflow"],"created_at":"2025-12-28T08:56:31.326Z","updated_at":"2026-04-02T17:52:03.365Z","avatar_url":"https://github.com/nh13.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# snakesee\n\n[![Language][language-badge]][language-link]\n[![Python][python-badge]][python-link]\n[![Code style][code-style-badge]][code-style-link]\n[![Type checked][type-check-badge]][type-check-link]\n[![License][license-badge]][license-link]\n\n[![Tests][tests-badge]][tests-link]\n[![codecov][codecov-badge]][codecov-link]\n[![Documentation][docs-badge]][docs-link]\n[![PyPI version][pypi-badge]][pypi-link]\n[![PyPI downloads][pypi-downloads-badge]][pypi-link]\n[![Bioconda][bioconda-badge]][bioconda-link]\n\n**A terminal UI for monitoring Snakemake workflows.**\n\nsnakesee provides a rich TUI dashboard for passively monitoring Snakemake workflows. It reads directly from the `.snakemake/` directory, requiring no special flags or configuration when running Snakemake.\n\n## Features\n\n- **Zero configuration** - Works on any existing workflow without modification\n- **Historical browsing** - Navigate through past workflow executions\n- **Time estimation** - Predicts remaining time from historical data\n- **Rich TUI** - Vim-style keyboard controls, filtering, and sorting\n- **Multiple layouts** - Full, compact, and minimal display modes\n\n## Why snakesee?\n\n| Tool | Approach | Requirements | Status |\n|------|----------|--------------|--------|\n| **snakesee** | Passive (reads `.snakemake/`) | None | Active |\n| [snkmt](https://github.com/cademirch/snkmt) | Active (logger plugin) | `--logger snkmt` + SQLite | Active |\n| [Panoptes](https://github.com/panoptes-organization/panoptes) | Active (WMS monitor) | `--wms-monitor` + server | Early dev |\n| [snakemake-terminal-monitor](https://github.com/nesi/snakemake-terminal-monitor) | Passive (reads logs) | Requires running workflow | Maintained |\n| [snk](https://github.com/Wytamma/snk) | CLI wrapper | Workflow installation | Active |\n| Built-in `--dag`/`--rulegraph` | Static visualization | Graphviz | Built-in |\n\n## Installation\n\n### pip (recommended)\n\n```bash\npip install snakesee\n```\n\n### pip with logo support\n\n```bash\npip install snakesee[logo]\n```\n\n### conda / mamba\n\n```bash\nconda install -c bioconda snakesee\n```\n\n## Usage\n\n### Watch a workflow in real-time\n\n```bash\n# In a workflow directory\nsnakesee watch\n\n# Or specify a path\nsnakesee watch /path/to/workflow\n```\n\n### Get a one-time status snapshot\n\n```bash\nsnakesee status\nsnakesee status /path/to/workflow\n```\n\n### Options\n\n```bash\nsnakesee watch --refresh 5.0      # Refresh every 5 seconds (default: 2.0)\nsnakesee watch --no-estimate      # Disable time estimation\nsnakesee status --no-estimate     # Status without ETA\n```\n\n## Time Estimation\n\nsnakesee predicts remaining workflow time using historical execution data from `.snakemake/metadata/`. The estimation uses multiple strategies depending on available data:\n\n### Estimation Methods\n\n| Method | When Used | Confidence |\n|--------|-----------|------------|\n| **Weighted** | Historical data available | High (0.5-0.9) |\n| **Simple** | No historical data, some jobs completed | Medium (0.3-0.7) |\n| **Bootstrap** | No jobs completed yet | Low (0.05) |\n\n### How It Works\n\n1. **Per-rule timing**: Historical execution times are tracked for each rule (e.g., `align`, `sort`, `index`)\n2. **Recency weighting**: Recent runs are weighted more heavily using exponential decay\n3. **Pending rule inference**: Assumes remaining jobs follow the same rule distribution as completed jobs\n4. **Parallelism adjustment**: Estimates concurrent job execution from historical completion rates\n\n### ETA Display Formats\n\n| Format | Meaning |\n|--------|---------|\n| `~5m` | High confidence estimate |\n| `3m - 8m` | Medium confidence, shows range |\n| `~10m (rough)` | Low confidence estimate |\n| `~15m (very rough)` | Very low confidence |\n| `unknown` | Insufficient data |\n\n### Weighting Strategies\n\nsnakesee supports two strategies for weighting historical timing data:\n\n#### Index-Based Weighting (Default)\n\nWeights runs by how many runs ago they occurred, regardless of actual time elapsed:\n- **Most recent run** has the highest weight\n- **Older runs** (by log index) progressively contribute less\n- **Default half-life**: 10 logs (after 10 runs, weight is halved)\n\nThis is ideal for **active development** where each pipeline run may fix issues:\n\n```bash\nsnakesee watch --weighting-strategy index --half-life-logs 10\n```\n\n#### Time-Based Weighting\n\nWeights runs by wall-clock time since each run:\n- **Recent runs** (within the last week) have the highest influence\n- **Default half-life**: 7 days (after 7 days, a run's weight is halved)\n\nThis is better for **stable pipelines** where old data should naturally age out:\n\n```bash\nsnakesee watch --weighting-strategy time --half-life-days 7\n```\n\nBoth strategies help adapt to:\n- Hardware changes (new machine, more cores)\n- Software updates (faster tool versions)\n- Pipeline improvements and bug fixes\n\n### Wildcard Conditioning\n\nWhen enabled, snakesee tracks timing separately for each wildcard value (e.g., `sample=A`, `sample=B`). This improves estimates when different inputs have significantly different runtimes.\n\n```bash\n# Enable via CLI flag\nsnakesee watch --wildcard-timing\n\n# Or toggle in TUI with 'w' key\n```\n\n**When to use**: Enable when your workflow processes inputs of varying sizes (e.g., genome samples, dataset batches) and execution times vary significantly between them.\n\n### Portable Timing Profiles\n\nExport timing data to share across machines or bootstrap new runs:\n\n```bash\n# Export profile from current workflow\nsnakesee profile-export\n\n# Export to a specific file\nsnakesee profile-export --output timing.json\n\n# Merge with existing profile (combine data)\nsnakesee profile-export --merge\n\n# View profile contents\nsnakesee profile-show .snakesee-profile.json\n\n# Use a profile for estimation\nsnakesee watch --profile timing.json\n```\n\nProfiles are auto-discovered: snakesee searches for `.snakesee-profile.json` in the workflow directory and parent directories.\n\n### Tool-Specific Progress Plugins\n\nsnakesee includes plugins that parse tool-specific log files to show real-time progress within running jobs. This is particularly useful for long-running bioinformatics tools.\n\n**Built-in plugins:**\n| Tool | Progress Detection |\n|------|-------------------|\n| **BWA** | Processed reads count |\n| **STAR** | Finished reads count |\n| **samtools sort** | Records processed |\n| **samtools index** | Records indexed |\n| **fastp** | Reads processed/passed |\n| **fgbio** | Records processed |\n\n**How it works:**\n1. When a job is running, snakesee searches for its log file\n2. Plugins detect the tool from rule name or log content\n3. Progress is extracted and displayed in the TUI\n\n**Creating custom plugins:**\n\nCreate a Python file in `~/.snakesee/plugins/` or `~/.config/snakesee/plugins/`:\n\n```python\n# ~/.snakesee/plugins/my_tool.py\nimport re\nfrom snakesee.plugins.base import ToolProgress, ToolProgressPlugin\n\nclass MyToolPlugin(ToolProgressPlugin):\n    @property\n    def tool_name(self) -\u003e str:\n        return \"mytool\"\n\n    def can_parse(self, rule_name: str, log_content: str) -\u003e bool:\n        return \"mytool\" in rule_name.lower()\n\n    def parse_progress(self, log_content: str) -\u003e ToolProgress | None:\n        # Parse your tool's log format\n        match = re.search(r\"Processed (\\d+) items\", log_content)\n        if match:\n            return ToolProgress(\n                items_processed=int(match.group(1)),\n                unit=\"items\"\n            )\n        return None\n```\n\nUser plugins are automatically discovered and loaded when snakesee starts.\n\n**Entry-point plugins (for package authors):**\n\nThird-party packages can register plugins via setuptools entry points. Add to your `pyproject.toml`:\n\n```toml\n[project.entry-points.\"snakesee.plugins\"]\nmy_tool = \"my_package.plugins:MyToolPlugin\"\n```\n\nEntry-point plugins are discovered automatically when the package is installed.\n\n### Enhanced Monitoring with Real-Time Events\n\nFor real-time event streaming (instead of log polling), you can enable event-based monitoring:\n\n#### Snakemake 9.0+ (Logger Plugin)\n\nInstall the optional Snakemake logger plugin:\n\n```bash\npip install snakemake-logger-plugin-snakesee\n```\n\nThen run Snakemake with the logger:\n\n```bash\nsnakemake --logger snakesee --cores 4\n```\n\n#### Snakemake 8.x (Log Handler Script)\n\nUse the built-in log handler script:\n\n```bash\nsnakemake --log-handler-script $(snakesee log-handler-path) --cores 4\n```\n\n\u003e **Note:** The log handler script is optimized for local execution where jobs start\n\u003e immediately after submission. For cluster/cloud executors (SLURM, AWS Batch, etc.),\n\u003e jobs shown as \"running\" may still be queued. For accurate queue tracking on clusters,\n\u003e use Snakemake 9+ with the logger plugin.\n\n#### Monitoring\n\nIn another terminal, monitor with snakesee:\n\n```bash\nsnakesee watch\n```\n\n**Benefits of real-time events:**\n\n| Feature | Log Parsing | Real-Time Events |\n|---------|-------------|------------------|\n| Job detection | Polling (delayed) | Immediate |\n| Start times | Approximate (log mtime) | Exact timestamp |\n| Durations | Calculated from logs | Precise from events |\n| Failed jobs | Pattern matching | Direct notification |\n\nReal-time events are optional - snakesee works without them using log parsing, and automatically uses events when available.\n\n### Workflow Status Detection\n\nsnakesee determines if a workflow is actively running by checking:\n\n1. **Lock files** exist in `.snakemake/locks/`\n2. **Incomplete markers** exist in `.snakemake/incomplete/` (jobs in progress)\n3. **Log file** was recently modified (within the stale threshold)\n\nIf lock files AND incomplete markers exist, the workflow is considered **RUNNING** regardless of log age. This handles very long-running jobs that don't update the log file.\n\nIf lock files exist but no incomplete markers, snakesee falls back to checking log freshness. The **stale threshold** defaults to **30 minutes** (1800 seconds). If the log hasn't been updated within this threshold, the workflow is considered interrupted (INCOMPLETE status).\n\n## TUI Keyboard Shortcuts\n\n### General\n\n| Key | Action |\n|-----|--------|\n| `q` | Quit |\n| `?` | Show help |\n| `p` | Pause/resume auto-refresh |\n| `e` | Toggle time estimation |\n| `w` | Toggle wildcard conditioning |\n| `r` | Force refresh |\n| `Ctrl+r` | Hard refresh (reload historical data) |\n\n### Refresh Rate\n\n| Key | Action |\n|-----|--------|\n| `+` / `-` | Fine adjust (±0.5s) |\n| `\u003c` / `\u003e` | Coarse adjust (±5s) |\n| `0` | Reset to default (1s) |\n| `G` | Set to minimum (0.5s, fastest) |\n\n### Layout \u0026 Filtering\n\n| Key | Action |\n|-----|--------|\n| `Tab` | Cycle layout (full/compact/minimal) |\n| `/` | Filter rules by name |\n| `n` / `N` | Next/previous filter match |\n| `Esc` | Clear filter, return to latest log |\n\n### Log History Navigation\n\n| Key | Action |\n|-----|--------|\n| `[` / `]` | View older/newer log (1 step) |\n| `{` / `}` | View older/newer log (5 steps) |\n\n### Table Sorting\n\n| Key | Action |\n|-----|--------|\n| `s` / `S` | Cycle sort table forward/backward |\n| `1-4` | Sort by column (press again to reverse) |\n\n### Modal Navigation (vim-style)\n\nsnakesee uses a two-mode navigation system for exploring jobs and logs:\n\n**Enter Table Mode:** Press `Enter` from the main view\n\n| Key | Action |\n|-----|--------|\n| `j` / `k` | Move down/up one row |\n| `g` / `G` | Jump to first/last row |\n| `Ctrl+d` / `Ctrl+u` | Half-page down/up |\n| `Ctrl+f` / `Ctrl+b` | Full-page down/up |\n| `h` / `l` | Switch to running/completions table |\n| `Tab` | Cycle between tables |\n| `Enter` | View selected job's log |\n| `Esc` | Exit table mode |\n\n**Log Viewing Mode:** Press `Enter` on a selected job\n\n| Key | Action |\n|-----|--------|\n| `j` / `k` | Scroll down/up one line |\n| `g` / `G` | Jump to start/end of log |\n| `Ctrl+d` / `Ctrl+u` | Half-page down/up |\n| `Ctrl+f` / `Ctrl+b` | Full-page down/up |\n| `Esc` | Return to table mode |\n\n## Development\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and guidelines.\n\n## Disclaimer\n\nThis codebase was written with the assistance of AI (Claude). All code has been reviewed and tested, but users should evaluate fitness for their use case.\n\n## License\n\n[MIT License](LICENSE) - Copyright (c) 2024 Fulcrum Genomics LLC\n\n[language-badge]: https://img.shields.io/badge/language-Python-blue\n[language-link]: https://www.python.org/\n[python-badge]: https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue\n[python-link]: https://www.python.org/\n[code-style-badge]: https://img.shields.io/badge/code%20style-ruff-000000\n[code-style-link]: https://github.com/astral-sh/ruff\n[type-check-badge]: https://img.shields.io/badge/type%20checked-mypy-blue\n[type-check-link]: https://mypy.readthedocs.io/\n[license-badge]: https://img.shields.io/badge/license-MIT-blue\n[license-link]: https://github.com/nh13/snakesee/blob/main/LICENSE\n[tests-badge]: https://github.com/nh13/snakesee/actions/workflows/tests.yml/badge.svg\n[tests-link]: https://github.com/nh13/snakesee/actions/workflows/tests.yml\n[codecov-badge]: https://codecov.io/gh/nh13/snakesee/graph/badge.svg\n[codecov-link]: https://codecov.io/gh/nh13/snakesee\n[docs-badge]: https://readthedocs.org/projects/snakesee/badge/?version=latest\n[docs-link]: https://snakesee.readthedocs.io/en/latest/\n[pypi-badge]: https://img.shields.io/pypi/v/snakesee\n[pypi-link]: https://pypi.org/project/snakesee/\n[pypi-downloads-badge]: https://img.shields.io/pypi/dm/snakesee\n[bioconda-badge]: https://img.shields.io/conda/vn/bioconda/snakesee\n[bioconda-link]: https://anaconda.org/bioconda/snakesee\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnh13%2Fsnakesee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnh13%2Fsnakesee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnh13%2Fsnakesee/lists"}