{"id":50914412,"url":"https://github.com/antont-cerebras/checkpoint-explorer","last_synced_at":"2026-06-16T13:30:39.242Z","repository":{"id":365224977,"uuid":"1271110641","full_name":"antont-cerebras/checkpoint-explorer","owner":"antont-cerebras","description":"CLI utility to inspect checkpoints (based on https://github.com/EricLBuehler/safetensors_explorer)","archived":false,"fork":false,"pushed_at":"2026-06-16T13:06:42.000Z","size":3190,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-06-16T13:10:16.469Z","etag":null,"topics":["checkpoint","gguf","hdf5","safetensors"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antont-cerebras.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-16T10:47:15.000Z","updated_at":"2026-06-16T13:08:30.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/antont-cerebras/checkpoint-explorer","commit_stats":null,"previous_names":["antont-cerebras/safetensors_explorer"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/antont-cerebras/checkpoint-explorer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antont-cerebras%2Fcheckpoint-explorer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antont-cerebras%2Fcheckpoint-explorer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antont-cerebras%2Fcheckpoint-explorer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antont-cerebras%2Fcheckpoint-explorer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antont-cerebras","download_url":"https://codeload.github.com/antont-cerebras/checkpoint-explorer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antont-cerebras%2Fcheckpoint-explorer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34408777,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["checkpoint","gguf","hdf5","safetensors"],"created_at":"2026-06-16T13:30:35.940Z","updated_at":"2026-06-16T13:30:39.237Z","avatar_url":"https://github.com/antont-cerebras.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Checkpoint Explorer\n\nAn interactive terminal-based explorer for [`safetensors`](https://huggingface.co/docs/safetensors) and [GGUF](https://huggingface.co/docs/hub/gguf) files, designed to help you visualize and navigate the structure of machine learning models.\n\n![Demo](demo.gif)\n\n## Features\n\n- 🔍 **Interactive browsing** of `safetensors` and GGUF file structures\n- 📁 **Hierarchical tree view** with expandable/collapsible groups\n- 🔎 **Fuzzy search** - instantly filter tensors with fuzzy matching using `/` key\n- 🔢 **Smart numeric sorting** for layer numbers (e.g., layer.0, layer.1, layer.2, ..., layer.10)\n- 📊 **Tensor details** including shape, data type, and size\n- 🔗 **Multi-file support** - automatically merges multiple files into a unified view\n- 📂 **Directory support** - explore entire model directories with automatic `safetensors` index detection\n- 🌟 **Glob pattern support** - use wildcards to select multiple files (e.g., `*.safetensors`, `model-*.gguf`)\n- 📏 **Human-readable sizes** (B, KB, MB, GB)\n- ⌨️ **Keyboard navigation** for smooth exploration\n- 🧠 **GGUF support** - view GGML format tensors with quantization types\n- 🧊 **HDF5 checkpoint support** (opt-in `--features hdf5`) - read Cerebras-style\n  `.h5`/`.hdf5` checkpoints, showing compression status and both the logical and\n  on-disk (compressed) sizes\n\n## Installation\n\n### Install\n```bash\ncargo install --git https://github.com/antont-cerebras/checkpoint-explorer\n```\n\n### Prerequisites\n- Rust (1.70 or later)\n\n### Build from source\n```bash\ngit clone https://github.com/antont-cerebras/checkpoint-explorer\ncd checkpoint-explorer\ncargo build --release\n```\n\n### HDF5 checkpoint support (optional)\nReading Cerebras-style HDF5 checkpoints is behind the `hdf5` feature, which is\noff by default so the standard build stays pure-Rust with no system\ndependencies. Enabling it bundles and statically links libhdf5 (requires a C\ntoolchain + `cmake`; the first build is slower):\n```bash\ncargo install --git \u003crepo-url\u003e --features hdf5\n# or from source:\ncargo build --release --features hdf5\n```\n\n## Usage\n\n### Basic usage\n```bash\n# Explore a single safetensors file\ncheckpoint-explorer model.safetensors\n\n# Explore a GGUF file\ncheckpoint-explorer model.gguf\n\n# Or if building from source\ncargo run -- model.safetensors\ncargo run -- model.gguf\n```\n\n### Directory exploration\n```bash\n# Explore all safetensors and GGUF files in a directory\ncheckpoint-explorer /path/to/model/directory\n\n# Recursively search subdirectories\ncheckpoint-explorer -r /path/to/models\n\n# The tool automatically detects and uses model.safetensors.index.json if present\ncheckpoint-explorer /path/to/huggingface/model\n```\n\n### Multi-file exploration\n```bash\n# Explore multiple files as a unified model\ncheckpoint-explorer model-00001-of-00003.safetensors model-00002-of-00003.safetensors model-00003-of-00003.safetensors\n\n# Mix safetensors and GGUF files\ncheckpoint-explorer model.safetensors model.gguf\n\n# Mix files and directories\ncheckpoint-explorer model.safetensors /path/to/additional/models\n```\n\n### Glob pattern support\n```bash\n# Use wildcards to select multiple files\ncheckpoint-explorer *.safetensors\n\n# Match files with specific patterns\ncheckpoint-explorer model-*.gguf\n\n# Match numbered checkpoint files\ncheckpoint-explorer checkpoint-[0-9]*.safetensors\n\n# Combine multiple patterns\ncheckpoint-explorer *.safetensors *.gguf\n\n# Mix glob patterns with explicit paths\ncheckpoint-explorer model.safetensors checkpoint-*.safetensors\n```\n\n### Keyboard Controls\n\n| Key | Action |\n|-----|--------|\n| `↑` / `↓` | Navigate up/down through the tree |\n| `Enter` / `Space` | Expand/collapse groups, view tensor details |\n| `/` | Enter search mode to filter tensors |\n| `Esc` | Exit search mode |\n| `q` | Quit the application (or exit search mode if active) |\n| `Ctrl+C` | Force quit |\n\n### Search Feature\n\nPress `/` to enter search mode and start typing to filter tensors by name. The search:\n- Uses **fuzzy matching** - find tensors even with typos or partial matches (e.g., \"attnproj\" will match \"attn.c_proj.weight\")\n- Searches **all tensors** - not just visible ones, regardless of collapsed groups\n- Shows results in a **flat list** with full tensor names\n- Sorts by **relevance** - best matches appear first\n\nPress `Enter` or `Esc` to exit search mode and return to the full tree view.\n\n## Example Output\n\n```\nSafeTensors Explorer - model.safetensors (1/1)\nUse ↑/↓ to navigate, Enter/Space to expand/collapse, q to quit\n================================================================================\n\n▼ 📁 transformer (123 tensors, 1.2 GiB)\n  ▼ 📁 h (32 layers, 120 tensors, 1.1 GiB)\n    ▼ 📁 0 (5 tensors, 45.2 MiB)\n      📄 attn.c_attn.weight [Float16, (4096, 3072), 25.2 MiB]\n      📄 attn.c_proj.weight [Float16, (1024, 4096), 8.4 MiB]\n      📄 ln_1.weight [Float16, (4096,), 8.2 KiB]\n      📄 mlp.c_fc.weight [Float16, (4096, 11008), 90.1 MiB]\n      📄 mlp.c_proj.weight [Float16, (11008, 4096), 90.1 MiB]\n    ▶ 📁 1 (5 tensors, 45.2 MiB)\n    ▶ 📁 2 (5 tensors, 45.2 MiB)\n    ...\n    ▶ 📁 31 (5 tensors, 45.2 MiB)\n  📄 ln_f.weight [Float16, (4096,), 8.2 KiB]\n  📄 wte.weight [Float16, (151936, 4096), 1.2 GiB]\n\nSelected: 1/342 | Scroll: 0\n```\n\n## How It Works\n\n1. **Path Resolution**: Automatically discovers `safetensors` files from files, directories, or `safetensors` index files\n2. **File Loading**: Loads one or more `safetensors` files and extracts tensor metadata\n3. **Tree Building**: Organizes tensors into a hierarchical structure based on their names (split by '.')\n4. **Smart Sorting**: Uses natural sorting to handle numeric components correctly\n5. **Interactive Display**: Renders the tree with expansion/collapse functionality\n6. **Tensor Details**: Shows detailed information when selecting individual tensors\n\n## Technical Details\n\n### Supported Formats\n- `safetensors` files (`.safetensors`)\n- GGUF files (`.gguf`) with GGML tensor types including quantized formats\n- HDF5 checkpoints (`.h5`/`.hdf5`) when built with `--features hdf5` — Cerebras\n  layout (URL-quoted tensor names as top-level datasets), with per-tensor\n  compression markers (e.g. `lz4`, `gzip`) and on-disk sizes\n- `safetensors` index files (`model.safetensors.index.json`)\n- Directory scanning with recursive search option\n- All tensor data types supported by the `safetensors` and GGML formats\n\n### Performance\n- Memory efficient: Only loads tensor metadata, not the actual tensor data\n- Fast startup: Optimized for quick exploration of large models\n- Responsive UI: Smooth navigation even with thousands of tensors\n\n## Dependencies\n\n- `safetensors` - For reading `safetensors` files\n- `gguf` - For reading GGUF files\n- `crossterm` - For terminal UI and keyboard input\n- `clap` - For command-line argument parsing\n- `anyhow` - For error handling\n- `serde_json` - For parsing `safetensors` index files\n- `glob` - For directory pattern matching\n\n## Contributing\n\nContributions are welcome! Please feel free to submit issues or pull requests.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantont-cerebras%2Fcheckpoint-explorer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantont-cerebras%2Fcheckpoint-explorer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantont-cerebras%2Fcheckpoint-explorer/lists"}