{"id":49418213,"url":"https://github.com/abd0r/quatrix","last_synced_at":"2026-04-29T04:09:06.625Z","repository":{"id":345589683,"uuid":"1177897399","full_name":"Abd0r/quatrix","owner":"Abd0r","description":"Quatrix — Q-Compass Architecture: novel neural architecture replacing attention with value-based navigation. Base for the Quasar model series.","archived":false,"fork":false,"pushed_at":"2026-04-28T04:16:04.000Z","size":1717,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-04-28T05:27:47.109Z","etag":null,"topics":["attention","deep-learning","language-model","multimodal","neural-architecture","pytorch","q-learning","reinforcement-learning","transformer","world-model"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Abd0r.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-10T13:38:47.000Z","updated_at":"2026-04-28T04:16:07.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Abd0r/quatrix","commit_stats":null,"previous_names":["abd0r/quatrix"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/Abd0r/quatrix","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Abd0r%2Fquatrix","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Abd0r%2Fquatrix/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Abd0r%2Fquatrix/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Abd0r%2Fquatrix/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Abd0r","download_url":"https://codeload.github.com/Abd0r/quatrix/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Abd0r%2Fquatrix/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32410024,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T03:46:11.172Z","status":"ssl_error","status_checked_at":"2026-04-29T03:37:55.317Z","response_time":110,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","deep-learning","language-model","multimodal","neural-architecture","pytorch","q-learning","reinforcement-learning","transformer","world-model"],"created_at":"2026-04-29T04:09:06.044Z","updated_at":"2026-04-29T04:09:06.612Z","avatar_url":"https://github.com/Abd0r.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Quatrix — Q-Compass Architecture\n\n\u003e *\"Where transformers retrieve by similarity, Quatrix navigates by value.\"*\n\n**Quatrix** replaces standard multi-head attention with **Q-Compass** — a sequence-mixing primitive grounded in the reinforcement-learning $Q$-function rather than in geometric similarity. The same block runs across text, vision, audio, world-state transition, and cross-field tasks (cancer mutation signatures, drug-response, survival).\n\nBuilt by **Syed Abdur Rehman Ali** ([@Abd0r](https://github.com/Abd0r)).\n\n---\n\n## Papers\n\nBoth PDFs are checked into [`Papers/`](./Papers).\n\n1. **Q-Compass: Grounding Sequence Mixing in Reinforcement Learning Navigation** — [`Papers/qcompass.pdf`](./Papers/qcompass.pdf) · [Zenodo (March 2026)](https://zenodo.org/records/19104202) · DOI [10.5281/zenodo.19104202](https://doi.org/10.5281/zenodo.19104202). Defines the routing primitive (3-projection, no $W_V$).\n2. **Quatrix: An Empirical Evaluation of Q-Compass and SAVO on Multimodal Sequence Modeling** — [`Papers/Quatrix.pdf`](./Papers/Quatrix.pdf) · [Zenodo (April 2026)](https://zenodo.org/records/19839718) · DOI [10.5281/zenodo.19839718](https://doi.org/10.5281/zenodo.19839718). Multi-seed evaluation at 60M / 120M / 180M, KV-cache analysis, cross-field cancer demonstration.\n\n---\n\n## Core Idea\n\n### Q-Compass (3-projection, no $W_V$)\n\n```\nstate  = x @ W_s          # \"Where am I?\"\naction = x @ W_a          # \"Where can I go?\"\nQ(s,a) = softmax(state @ action.T / sqrt(r))\noutput = W_o(Q(s,a) @ x)  # gather raw x — no W_V\n```\n\nThree projections ($W_s, W_a, W_o$). Value-based routing — *\"in state s, how valuable is attending to position a?\"*\n\n### SAVO (4-projection variant: $Q$-value content)\n\nSAVO reintroduces a $V$, but the $V$ projects the state⊙action product (a $Q$-value), not the raw input:\n\n```\nqval    = state ⊙ action                  # ∈ R^r, the Q-value vector\ncontent = qval @ W_c                      # ∈ R^H, projected back up\noutput  = W_o(Q(s,a) @ content)\n```\n\nFour projections ($W_s, W_a, W_c, W_o$). Unlike standard attention's $W_V$ (linear map of raw $x$), SAVO's $W_c$ projects a $Q$-value — the $W_V$-free property is preserved at the *raw-input* level. The cost: $+rH$ parameters per block.\n\n---\n\n## Headline Empirical Results (paper §5.2 + §5.10)\n\n| Comparison | Number | Recipe |\n|---|---|---|\n| SAVO vs rank-matched MHA, 60M (4-seed paired) | $+12.33 \\pm 0.87$ ppl, $p = 7.6\\!\\times\\!10^{-4}$ | 10k steps, identical hyperparameters |\n| SAVO vs full-rank standard MHA, 60M (val) | $+5.79$ ppl above (worse) | full-rank MHA is parameter-undertrained at 10k steps |\n| Rank-matched MHA vs full-rank MHA, 60M | $-6.54$ ppl below (better) | rank-matched 1×-attn-block converges; full-rank 8×-attn-block does not |\n| KV-cache @ $r{=}H/8$ vs MHA | **0.125×** (matches MQA) | structural — content path is rank-$r$ by construction |\n| KV-cache @ $r{=}H/16$ vs MHA | **0.0625×** (16× smaller) | $\\le 1.6$ ppl penalty vs $r{=}H/8$ |\n| Cross-field (cancer Phase 1–4) | parity within $\\sim$5% of specialist baselines | same SAVO block, only I/O changes |\n\n---\n\n## Architecture\n\n```\nQuatrixLM (language model)\n├── Token + Positional Embeddings\n├── N × QuatrixBlock\n│   ├── LayerNorm → QCompass (causal) → residual\n│   └── LayerNorm → FFN (GELU) → residual\n├── LayerNorm\n└── Output Head (tied to embeddings)\n\nQuatrixVision (image encoder)\n├── Conv2d patch embedding (16×16 patches → 196 patches per 224×224 image)\n├── Positional embeddings\n├── M × QCompassBi blocks (bidirectional, no causal mask)\n└── Linear projection → LM hidden dim\n\nQuatrixAudio (audio encoder)\n├── Mel-spectrogram patch embedding (16×16 freq×time patches)\n├── 3 × QCompassBi blocks\n└── Linear projection → LM hidden dim\n\nQuatrixWorld (world-model plugin)\n├── StateEncoder: QCompassBi aggregates a frame patch sequence → state vector\n├── ActionHead: predicts action distribution from state\n├── TransitionModel: 9–10 × QCompassBi blocks, predicts ŝ' = f(s, a)\n└── RewardHead (optional): scalar value for RL fine-tuning\n\nQuatrixWorldGenerative (frame-prediction world model)\n└── Same Q-Compass block class, predicts the next FRAME (pixels), not just a latent state.\n\nQuatrixCancerModel (cancer mutation-signature model)\n└── SAVO stack ($H{=}384$, $r{=}48$, $h{=}6$) over SBS96 context vectors → softmax over signatures or cancer types.\n\nQuatrixEditModel (gene-editing outcome predictor)\n└── Architectural mirror of QuatrixWorldGenerative, applied to CRISPR edit outcomes.\n\nTransformerLM (rank-matched MHA baseline, paper §5.2)\n└── Standard 4-projection QKVO attention with all projections at rank r — apples-to-apples controlled ablation against SAVO.\n```\n\n---\n\n## Repository Layout\n\n```\nquatrix/                         (repo root)\n├── Papers/                      ← both papers\n│   ├── Quatrix.pdf              ← April 2026 empirical paper (this repo)\n│   └── qcompass.pdf             ← March 2026 original primitive paper\n├── src/quatrix/                 ← Python package (importable as `import quatrix`)\n│   ├── __init__.py              ← public API (QuatrixLM, QuatrixConfig, ...)\n│   ├── config.py                ← QuatrixConfig dataclass\n│   ├── model.py                 ← QCompass, QuatrixBlock, QuatrixLM (SAO + SAVO)\n│   ├── vision.py                ← QCompassBi, VisionEncoder\n│   ├── audio.py                 ← AudioEncoder, waveform_to_mel\n│   ├── world.py                 ← WorldModel + StateEncoder + TransitionModel\n│   ├── world_generative.py      ← QuatrixWorldGenerative (frame-prediction)\n│   ├── cancer_model.py          ← QuatrixCancerModel (paper §7 Phase 1–4)\n│   ├── edit_model.py            ← QuatrixEditModel (CRISPR-edit outcomes)\n│   ├── transformer_lm.py        ← TransformerLM rank-matched MHA baseline (paper §5.2)\n│   └── train.py                 ← python -m quatrix.train demo loop\n├── pyproject.toml\n├── requirements.txt\n├── LICENSE\n└── README.md\n```\n\n---\n\n## Modality Support\n\n| Modality | Module | Block class | Attention mode |\n|----------|--------|-------------|----------------|\n| Text | `QuatrixLM` | `QCompass` | causal |\n| Vision | `VisionEncoder` | `QCompassBi` | bidirectional |\n| Audio | `AudioEncoder` | `QCompassBi` | bidirectional |\n| World (latent) | `WorldModel` | `QCompassBi` | bidirectional |\n| World (generative) | `QuatrixWorldGenerative` | `QCompassBi` | bidirectional |\n| Cancer signatures | `QuatrixCancerModel` | `QCompassBi` (MH-QVC) | bidirectional |\n| Gene-editing | `QuatrixEditModel` | `QCompassBi` | bidirectional |\n\n---\n\n## Quick Start\n\n```bash\npip install quatrix\n```\n\n```python\nfrom quatrix import QuatrixLM, QuatrixConfig\nimport torch\n\n# Text only\ncfg = QuatrixConfig(vocab_size=50257, hidden_size=512, num_layers=7,\n                    max_seq_len=5120, q_rank=64)\nmodel = QuatrixLM(cfg)\ninput_ids = torch.randint(0, 50257, (1, 10))\nout = model(input_ids)\nlogits = out['logits']  # [B, L, vocab_size]\n\n# Text + Vision\ncfg = QuatrixConfig(vocab_size=50257, hidden_size=512, num_layers=7,\n                    max_seq_len=5120, q_rank=64, use_vision=True)\nmodel = QuatrixLM(cfg)\npixel_values = torch.randn(1, 3, 224, 224)\nout = model(input_ids, pixel_values=pixel_values)\n\n# Text + Vision + Audio\ncfg = QuatrixConfig(vocab_size=50257, hidden_size=512, num_layers=7,\n                    max_seq_len=5120, q_rank=64, use_vision=True, use_audio=True)\nmodel = QuatrixLM(cfg)\nmel = torch.randn(1, 1, 80, 3000)\nout = model(input_ids, pixel_values=pixel_values, mel=mel)\n\n# World Model\nfrom quatrix import WorldModel\nworld = WorldModel(lm_hidden=512, action_dim=256)\nhidden_states = model.get_hidden_states(input_ids)\nstate, action_logits, next_state, reward = world(hidden_states)\n```\n\n### Training\n\n```bash\n# Quick demo — TinyShakespeare, CPU/GPU\npython -m quatrix.train\n\n# Custom config\npython -m quatrix.train --steps 2000 --hidden 512 --layers 7\npython -m quatrix.train --data myfile.txt\n```\n\n---\n\n## Roadmap\n\n| Project | Description | Status |\n|---|---|---|\n| Q-Compass v1 | Routing primitive, 3-projection | Published (Zenodo) |\n| Quatrix v1 (this repo) | SAVO 4-projection + multimodal evaluation + KV-cache analysis + cross-field demo | Empirical paper out |\n| **NanoG1** | Cancer foundation model with mid-CoT hypothetical simulation, building on the Phase 1–4 setup in `cancer_model.py` | Future work |\n\n---\n\n## Citation\n\nIf you use Quatrix or Q-Compass in your work, please cite:\n\n```bibtex\n@misc{ali2026qcompass,\n  author       = {Syed Abdur Rehman Ali},\n  title        = {Q-Compass: Grounding Sequence Mixing in Reinforcement Learning Navigation},\n  year         = {2026},\n  month        = {March},\n  howpublished = {Zenodo},\n  doi          = {10.5281/zenodo.19104202},\n  url          = {https://zenodo.org/records/19104202}\n}\n\n@misc{ali2026quatrix,\n  author       = {Syed Abdur Rehman Ali},\n  title        = {Quatrix: An Empirical Evaluation of Q-Compass and SAVO on Multimodal Sequence Modeling},\n  year         = {2026},\n  month        = {April},\n  howpublished = {Zenodo},\n  doi          = {10.5281/zenodo.19839718},\n  url          = {https://zenodo.org/records/19839718}\n}\n```\n\n---\n\n## Author\n\n**Syed Abdur Rehman Ali**\n\n[![GitHub](https://img.shields.io/badge/GitHub-Abd0r-181717?style=for-the-badge\u0026logo=github\u0026logoColor=white)](https://github.com/Abd0r)\n[![HuggingFace](https://img.shields.io/badge/🤗%20HuggingFace-Abd0r-FFD21E?style=for-the-badge\u0026logoColor=black)](https://huggingface.co/Abd0r)\n[![X](https://img.shields.io/badge/X%20%2F%20Twitter-SyedAbdurR2hman-000000?style=for-the-badge\u0026logo=x\u0026logoColor=white)](https://x.com/SyedAbdurR2hman)\n\n---\n\n## License\n\nOpenRAIL-M — open use with behavioral restrictions (no military use, no mass surveillance). See `LICENSE` for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabd0r%2Fquatrix","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabd0r%2Fquatrix","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabd0r%2Fquatrix/lists"}