{"id":45015160,"url":"https://github.com/ginozza/shrew","last_synced_at":"2026-02-19T01:00:46.847Z","repository":{"id":339103477,"uuid":"1157744841","full_name":"ginozza/shrew","owner":"ginozza","description":"A polyglot neural engine for modular, language-agnostic deep learning with decoupled execution.","archived":false,"fork":false,"pushed_at":"2026-02-18T03:44:22.000Z","size":1102,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-02-18T07:14:46.381Z","etag":null,"topics":["deep-learning","dls","gpu","machine-learning","python","rust","tensor","transformer"],"latest_commit_sha":null,"homepage":"https://shrew.ink","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ginozza.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/funding.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["ginozza"]}},"created_at":"2026-02-14T08:17:51.000Z","updated_at":"2026-02-18T04:51:10.000Z","dependencies_parsed_at":null,"dependency_job_id":"9b2f5ec0-3ec4-4255-a851-5ec5e4cf2400","html_url":"https://github.com/ginozza/shrew","commit_stats":null,"previous_names":["ginozza/shrew"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/ginozza/shrew","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ginozza%2Fshrew","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ginozza%2Fshrew/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ginozza%2Fshrew/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ginozza%2Fshrew/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ginozza","download_url":"https://codeload.github.com/ginozza/shrew/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ginozza%2Fshrew/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29600340,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T00:59:38.239Z","status":"ssl_error","status_checked_at":"2026-02-19T00:59:36.936Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","dls","gpu","machine-learning","python","rust","tensor","transformer"],"created_at":"2026-02-19T01:00:27.703Z","updated_at":"2026-02-19T01:00:46.829Z","avatar_url":"https://github.com/ginozza.png","language":"Rust","funding_links":["https://github.com/sponsors/ginozza"],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/shrew_banner.svg\" alt=\"Shrew\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/ginozza/shrew/actions/workflows/ci.yml\"\u003e\u003cimg src=\"https://img.shields.io/github/actions/workflow/status/ginozza/shrew/ci.yml?branch=master\u0026label=CI\" alt=\"CI\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/ginozza/shrew/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-Apache--2.0-blue.svg\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://crates.io/crates/shrew\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/shrew.svg\" alt=\"Crates.io\"\u003e\u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/badge/rust-1.75%2B-orange.svg\" alt=\"MSRV\"\u003e\n\u003c/p\u003e\n\n---\n\nShrew is a modular deep learning framework written from scratch in Rust. It provides a tensor system with automatic differentiation, neural network layers, optimizers, a CUDA GPU backend, and a declarative intermediate representation (`.sw`) for language-agnostic model specification.\n\nModels defined in `.sw` files can be trained from Python (via PyO3 bindings) or Rust, and deployed with zero transpilation across platforms.\n\n## Components\n\n| Crate | Description |\n|-------|-------------|\n| **shrew-core** | `Tensor\u003cB\u003e`, `Shape`, `DType`, `Layout`, `Backend` trait, reverse-mode autograd, dynamic symbolic shapes |\n| **shrew-cpu** | CPU backend: SIMD matmul via `gemm` (AVX2/AVX-512/FMA), parallel ops via `rayon`, broadcasting |\n| **shrew-cuda** | NVIDIA GPU backend: cuBLAS matmul, custom PTX kernels, memory pool, mixed-precision F16/BF16 |\n| **shrew-nn** | Neural network layers: Linear, Conv1d/2d, RNN/LSTM/GRU, MultiHeadAttention, Transformer, BatchNorm, LayerNorm, losses |\n| **shrew-optim** | Optimizers (SGD, Adam, AdamW, RAdam, RMSProp), LR schedulers, gradient clipping, EMA |\n| **shrew-ir** | `.sw` format: lexer, parser, AST, Graph IR, lowering, validation, shape inference, optimization passes |\n| **shrew-data** | `Dataset` trait, `DataLoader`, MNIST, image transforms, async prefetch loader |\n| **shrew** | Facade crate: executor, JIT compiler, trainer, distributed training, quantization, ONNX, profiling, checkpoints |\n| **shrew-python** | Python bindings via PyO3 with NumPy interop |\n| **shrew-cli** | CLI tools: `shrew dump`, `validate`, `bench`, `info` |\n\n## Technical Features\n\n### Backend-Agnostic Tensor System\n\n`Tensor\u003cB\u003e` is generic over `Backend`. The same tensor code runs on CPU and GPU without changes. Supported dtypes: `F16`, `BF16`, `F32`, `F64`, `U8`, `U32`, `I64`.\n\n```rust\nuse shrew::prelude::*;\n\nlet dev = CpuDevice;\nlet a = CpuTensor::randn((3, 4), DType::F32, \u0026dev)?;\nlet b = CpuTensor::randn((4, 5), DType::F32, \u0026dev)?;\nlet c = a.matmul(\u0026b)?;  // [3,4] × [4,5] → [3,5]\n```\n\nOperations: `add`, `sub`, `mul`, `div` (with NumPy-style broadcasting), `neg`, `abs`, `exp`, `log`, `sqrt`, `square`, `sin`, `cos`, `relu`, `sigmoid`, `tanh`, `gelu`, `silu`, `softmax`, `log_softmax`, `matmul`, `reshape`, `transpose`, `narrow`, `unsqueeze`, `expand`, `cat`, `chunk`, `index_select`, `sum`, `mean`, `max`, `min`, `argmax`, `argmin`, `var`, comparisons (`eq`, `ne`, `gt`, `ge`, `lt`, `le`), `to_dtype`.\n\n### Reverse-Mode Automatic Differentiation\n\nEager autograd — every op records its computational graph. `backward()` does topological sort and applies the chain rule. Gradient paths cover all binary/unary ops, reductions, matmul, reshape, transpose, narrow, affine, contiguous, cat, and index_select.\n\n```rust\nlet w = CpuTensor::randn((3, 3), DType::F64, \u0026dev)?.set_variable();\nlet x = CpuTensor::randn((2, 3), DType::F64, \u0026dev)?;\nlet loss = x.matmul(\u0026w)?.sum_all()?;\nlet grads = loss.backward()?;\nlet dw = grads.get(\u0026w).unwrap();  // ∂loss/∂w\n```\n\n### Neural Network Layers\n\nAll layers implement `Module::forward()` and are generic over `Backend`:\n\n| Category | Layers |\n|----------|--------|\n| Dense | `Linear` |\n| Convolution | `Conv1d`, `Conv2d`, `MaxPool2d`, `AvgPool2d`, `AdaptiveAvgPool2d` |\n| Recurrent | `RNNCell`, `RNN`, `LSTMCell`, `LSTM`, `GRUCell`, `GRU` |\n| Attention | `MultiHeadAttention`, `TransformerBlock` |\n| Normalization | `BatchNorm2d`, `LayerNorm`, `GroupNorm`, `RMSNorm` |\n| Embedding | `Embedding` |\n| Regularization | `Dropout`, `Flatten`, `Sequential` |\n| Activations | `ReLU`, `GeLU`, `SiLU`, `LeakyReLU`, `ELU`, `Mish` |\n| Losses | `mse_loss`, `cross_entropy_loss`, `l1_loss`, `smooth_l1_loss`, `bce_loss`, `bce_with_logits_loss`, `nll_loss` |\n\n### Optimizers and Schedulers\n\n| Optimizers | Schedulers |\n|------------|-----------|\n| `SGD` (momentum, weight decay) | `StepLR`, `ExponentialLR`, `LinearLR` |\n| `Adam`, `AdamW`, `RAdam` | `CosineAnnealingLR`, `CosineWarmupLR` |\n| `RMSProp` | `ReduceLROnPlateau` |\n\nUtilities: `clip_grad_norm`, `clip_grad_value`, `grad_norm`, `GradAccumulator`, `EMA`.\n\n### CUDA GPU Backend\n\nFeature-gated backend using `cudarc`. cuBLAS for matrix multiplication, custom PTX kernels for elementwise, reduction, broadcast, and cast operations. Includes a memory pool with allocation reuse.\n\n```bash\ncargo build -p shrew --features cuda\n```\n\nMixed-precision training: `MixedPrecisionTrainer` with dynamic loss scaling, automatic F32↔F16/BF16 casting via `to_dtype`.\n\n### `.sw` Intermediate Representation\n\nDeclarative, text-based model specification — separates model architecture from runtime execution:\n\n```sw\n@model { name: \"TinyGPT\"; }\n\n@config {\n    d_model: 256;\n    n_heads: 4;\n    d_ff: 256 * 4;   // constant folding → 1024\n}\n\n@graph Forward {\n    input tokens: Tensor\u003c[Batch, SeqLen], i64\u003e;\n    param wte: Tensor\u003c[50257, 256], f32\u003e { init: \"normal(0, 0.02)\"; };\n    param wpe: Tensor\u003c[512, 256], f32\u003e   { init: \"normal(0, 0.02)\"; };\n\n    node tok_emb  { op: embedding(wte, tokens); };\n    node pos_emb  { op: embedding(wpe, positions); };\n    node h        { op: tok_emb + pos_emb; };\n    node tf_out   { op: repeat(4) { transformer_block(h, n_heads: 4); }; };\n    node ln_out   { op: layer_norm(tf_out, ln_w, ln_b, eps: 1e-5); };\n    node logits   { op: matmul(ln_out, transpose(wte)); };\n    output logits;\n}\n\n@training {\n    loss: cross_entropy;\n    optimizer: { type: \"AdamW\"; lr: 3e-4; weight_decay: 0.1; }\n    epochs: 20;\n    batch_size: 64;\n}\n```\n\nPipeline: source → **Lexer** → tokens → **Parser** → AST → **Lowering** → Graph IR → **Validate** → **Shape inference** → **Optimize** (DCE, CSE, constant folding, operator fusion, identity elimination).\n\n### JIT Compilation\n\n`JitExecutor` compiles IR graphs into a flat instruction tape with pre-allocated memory slots and value lifetime tracking. No re-interpretation of the graph at runtime.\n\n```rust\nuse shrew::exec::jit::load_jit;\n\nlet executor = load_jit::\u003cCpuBackend\u003e(sw_source, CpuDevice, config)?;\nlet result = executor.run(\"Forward\", \u0026inputs)?;\n```\n\n### Dynamic Symbolic Shapes\n\n`SymDim` (Fixed/Symbolic/Dynamic), `SymbolicShape`, `ShapeEnv`, and `ShapeGuard` bridge symbolic IR shapes with runtime concrete shapes. Supports shape unification, matching, and broadcasting.\n\n### Distributed Training\n\n| Component | Description |\n|-----------|-------------|\n| `DataParallel` | Batch splitting across workers with output concatenation |\n| `PipelineParallel` | GPipe-style micro-batch pipelining |\n| `MixedPrecisionTrainer` | Dynamic loss scaling for FP16/BF16 |\n| `reduce_gradients` | All-reduce gradient synchronization |\n\n### Quantization\n\nINT8/INT4 post-training quantization (symmetric/asymmetric, per-tensor/per-channel). `QuantizedLinear` for dequantize-on-the-fly inference.\n\n### ONNX Interop\n\nExport/import ONNX models (opset 17) with a built-in minimal protobuf encoder/decoder (zero external dependencies).\n\n### Profiling\n\n`Profiler` with named timing events, `MemoryTracker`, `ModelSummary`, `benchmark_forward`, `benchmark_forward_backward`.\n\n### Serialization\n\n| Format | Description |\n|--------|-------------|\n| `.shrew` | Native binary checkpoint (`save_tensors` / `load_tensors`) |\n| Safetensors | HuggingFace-compatible (`save_safetensors` / `load_safetensors`) |\n| ONNX | Open Neural Network Exchange (`export_weights` / `load_onnx_weights`) |\n\n## Installation\n\n### Rust\n\n```toml\n# Cargo.toml\n[dependencies]\nshrew = \"0.1\"\n```\n\nWith CUDA support:\n\n```toml\n[dependencies]\nshrew = { version = \"0.1\", features = [\"cuda\"] }\n```\n\nOr directly from GitHub:\n\n```toml\n[dependencies]\nshrew = { git = \"https://github.com/ginozza/shrew\" }\n```\n\n### CLI\n\n```bash\ncargo install shrew-cli\n```\n\nOr from source:\n\n```bash\ncargo install --git https://github.com/ginozza/shrew shrew-cli\n```\n\nThis installs the `shrew` binary with commands: `dump`, `validate`, `bench`, `info`.\n\n### Python\n\n```bash\npip install shrew-python\n```\n\nOr build from source:\n\n```bash\ngit clone https://github.com/ginozza/shrew\ncd shrew\npip install maturin\nmaturin develop --release\n```\n\n```python\nimport shrew_python as shrew\n\nt = shrew.tensor([1.0, 2.0, 3.0])\nprint(t)\n```\n\n### From Source (full workspace)\n\n```bash\ngit clone https://github.com/ginozza/shrew\ncd shrew\ncargo build --workspace\ncargo test --workspace\n```\n\nRequirements:\n- Rust 1.75+ (edition 2021)\n- Python 3.9+ (for Python bindings)\n- NVIDIA CUDA Toolkit (for GPU backend only)\n\n## Getting Started\n\n### Tensor operations and autograd\n\n```rust\nuse shrew::prelude::*;\n\nfn main() -\u003e shrew::Result\u003c()\u003e {\n    let dev = CpuDevice;\n\n    // Broadcasting: [3,1] + [1,2] → [3,2]\n    let a = CpuTensor::from_f64_slice(\u0026[1.0, 2.0, 3.0], (3, 1), DType::F64, \u0026dev)?;\n    let b = CpuTensor::from_f64_slice(\u0026[10.0, 20.0], (1, 2), DType::F64, \u0026dev)?;\n    let c = a.add(\u0026b)?;\n\n    // Transformer forward pass\n    let block = TransformerBlock::\u003cCpuBackend\u003e::new(64, 4, 256, true, DType::F64, \u0026dev)?;\n    let x = CpuTensor::rand((2, 10, 64), DType::F64, \u0026dev)?;\n    let y = block.forward(\u0026x)?;  // [2,10,64]\n\n    // Autograd\n    let w = CpuTensor::rand((3, 3), DType::F64, \u0026dev)?.set_variable();\n    let input = CpuTensor::rand((2, 3), DType::F64, \u0026dev)?;\n    let loss = input.matmul(\u0026w)?.sum_all()?;\n    let grads = loss.backward()?;\n\n    Ok(())\n}\n```\n\n### Executing a `.sw` model\n\n```rust\nuse shrew::prelude::*;\nuse shrew::exec::{load_program, RuntimeConfig};\n\nlet src = r#\"\n@model { name: \"MLP\"; }\n@graph Forward {\n    input x: Tensor\u003c[2, 4], f32\u003e;\n    param w: Tensor\u003c[4, 3], f32\u003e { init: \"normal(0, 0.1)\"; };\n    node out { op: softmax(matmul(x, w)); };\n    output out;\n}\n\"#;\n\nlet config = RuntimeConfig::default().with_dtype(DType::F32);\nlet exec = load_program::\u003cCpuBackend\u003e(src, CpuDevice, config)?;\n\nlet x = CpuTensor::rand((2, 4), DType::F32, \u0026CpuDevice)?;\nlet mut inputs = std::collections::HashMap::new();\ninputs.insert(\"x\".to_string(), x);\n\nlet result = exec.run(\"Forward\", \u0026inputs)?;\nlet probs = result.get(\"out\").unwrap();\nassert_eq!(probs.dims(), \u0026[2, 3]);\n```\n\n### Quantization\n\n```rust\nuse shrew::prelude::*;\n\nlet model = Linear::\u003cCpuBackend\u003e::new(256, 128, true, DType::F32, \u0026CpuDevice)?;\nlet config = QuantConfig::int8_per_channel();\nlet quantized = quantize_named_parameters::\u003cCpuBackend\u003e(\u0026model, \u0026config)?;\n```\n\n### Benchmarking\n\n```rust\nuse shrew::prelude::*;\n\nlet model = Linear::\u003cCpuBackend\u003e::new(512, 256, true, DType::F32, \u0026CpuDevice)?;\nlet result = benchmark_forward(\n    \u0026model,\n    || Tensor::\u003cCpuBackend\u003e::rand((32, 512), DType::F32, \u0026CpuDevice).unwrap(),\n    32, 5, 100,\n)?;\nprintln!(\"{}\", result);\n```\n\n## Build \u0026 Test\n\n```bash\ncargo build --workspace           # Build all crates\ncargo test --workspace            # Run all tests (~600+)\ncargo clippy --workspace          # Lint\ncargo fmt --all --check           # Format check\ncargo doc --workspace --no-deps   # Generate documentation\n```\n\n### Examples\n\n```bash\ncargo run -p example-linear-regression\ncargo run -p example-mlp-xor\ncargo run -p mnist-example                    # Requires MNIST data download\ncargo run -p mnist-cnn-example\ncargo run --release -p char-gpt-example       # Char-level GPT on Shakespeare\ncargo run -p example-rnn-sequence\ncargo run --release -p example-bench-ops      # CPU performance benchmarks\n```\n\n### CPU Performance (release mode)\n\n| Operation | Size | Time |\n|-----------|------|------|\n| matmul | 256×256 × 256×256 | ~370 µs |\n| matmul | 512×512 × 512×512 | ~3.5 ms |\n| add | 1M elements | ~1.3 ms |\n| linear forward | [64,512]×[512,512]+bias | ~3.9 ms |\n\n## Dependencies\n\n| Crate | Purpose |\n|-------|---------|\n| `gemm` | SIMD-accelerated matmul (auto AVX2/AVX-512/FMA) |\n| `rayon` | Parallel iteration for batched ops |\n| `half` | F16/BF16 with num-traits |\n| `cudarc` | CUDA driver/runtime, cuBLAS (optional) |\n| `pyo3` / `numpy` | Python bindings (optional) |\n| `num-traits` | Numeric trait bounds |\n| `rand` / `rand_distr` | Random initialization |\n| `thiserror` | Error types |\n| `serde_json` | Checkpoint metadata |\n\n## Contributing\n\nBug fixes are welcome without prior discussion. For new features or architectural changes, please open an issue first. See the [CHANGELOG](CHANGELOG.md) for release history.\n\n## License\n\nApache-2.0. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fginozza%2Fshrew","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fginozza%2Fshrew","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fginozza%2Fshrew/lists"}