https://github.com/TheApeMachine/caramba
https://github.com/TheApeMachine/caramba
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/TheApeMachine/caramba
- Owner: TheApeMachine
- License: other
- Created: 2026-05-10T17:54:03.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2026-06-09T12:53:49.000Z (2 months ago)
- Last Synced: 2026-06-09T14:23:27.470Z (2 months ago)
- Language: TypeScript
- Size: 37.8 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Agents: AGENTS.md
Awesome Lists containing this project
README
# 🌈 caramba
**A substrate for A.I. research.**
**Caramba** is a comprehensive machine learning research stack built to guide you through the entire lifecycle of A.I. development. Whether you want to rapidly prototype a new concept or dive deep into low-level hardware optimization, Caramba provides a dedicated environment tailored to your exact workflow.
Operating on the core philosophy that **a manifest is a model**, Caramba allows you to define complex architectures entirely via YAML files instead of writing code. It acts as a complete laboratory, seamlessly taking you from your initial idea to a heavily optimized, fully profiled pipeline.
**Core Capabilities:**
* 📝 **Fully Manifest-Driven:** Declare your topology in simple YAML. Caramba goes far beyond standard layers and operations, allowing you to easily express sophisticated, non-standard mathematical primitives and advanced custom architectures.
* 🚀 **Flexibility & Optimization:** Dedicated to high performance, Caramba gives you the tools to choose your level of abstraction. Move fast to iterate on high-level ideas, or drop down for granular, low-level control over compute and memory.
* 🔬 **Sophisticated Inspection:** Deeply understand your network's behavior. Caramba is equipped with advanced inspection and profiling tooling, bringing clarity to every step of the end-to-end research process.
* 🤖 **Integrated A.I. Collaboration:** Supercharge your workflow with a built-in A.I. assistant and virtual research team. Caramba is built from the ground up to support both human team collaboration and agentic brainstorming.
* 🔐 **Zero-Compromise Privacy:** Working with proprietary or sensitive data? Caramba can operate entirely in an optional "local-only" mode, ensuring your research and intellectual property never leave your secure environment.
## ✨ Features
- [x] Compute Primitives
- [x] Activation (ReLU, LeakyReLU, SELU, Sigmoid, Tanh, GeLU, Swish, SwiGLU)
- [x] Attention (SDPA, MQA, GQA, sliding window, softmax)
- [x] Convolution (Conv1D, Conv2D, Conv3D, ConvTranspose2D)
- [x] Embedding (token, RoPE, ALiBi, tied)
- [x] Math (matmul, add/mul/pow/atan2/mod, exp/log, rmsnorm, layernorm, groupnorm, softmax, logsumexp, dropout, sin, cos)
- [x] Pooling (avg, max, adaptive avg, adaptive max)
- [x] Projection (linear, fused QKV, tied embedding)
- [x] Shape (reshape, transpose, gather, scatter, where, masked fill, concat, split, view_as_heads, merge_heads, last_token, nearest upsample)
- [x] Masking (causal mask, apply mask)
- [x] Active Inference (free energy, expected free energy, belief update, precision weighting)
- [x] Energy-Based Model Blocks (Boltzmann distribution, EBM free energy, Langevin step, contrastive phase)
- [x] Causal Inference (do-calculus, backdoor, frontdoor, CATE, IV, counterfactual, DAG factorization)
- [x] Hawkes Process (intensity, kernel matrix, simulate, log-likelihood)
- [x] Markov Blanket (partition, mutual information, internal/active flow)
- [x] Predictive Coding (prediction, prediction error, representation/weight updates)
- [x] VSA (bind, bundle, permute, inverse permute, similarity)
- [x] Multiple Compute Backends
- [x] CPU (Go native; **amd64** and **arm64** only — 32-bit `GOARCH=386` is not supported)
- [x] SIMD/Assembly
- [ ] AVX-512 (amd64)
- [x] AVX2 (amd64)
- [x] SSE2 (amd64)
- [x] NEON (arm64)
- [x] CUDA
- [x] METAL
- [x] XLA
- [x] Optimizers (SGD, Adam, AdamW, AdaMax, AdaGrad, AdaDelta, RMSProp, Lion, LARS, LAMB, L-BFGS, Hebbian)
- [ ] Training Models
- [ ] Fine-tuning Models
- [x] Manifest Compiler (verify, canonicalize, CSE, algebraic simplify, fusion, DCE, memory planning, cost scheduling)
- [x] SafeTensors architecture manifests (`from_safetensors`, config-driven registry lookup, direct tensor binding)
- [x] Hugging Face Hub Asset Resolver (revision-pinned, content-addressed, Xet CAS)
- [x] Provenance Ledger (signed by `pkg/notary`)
- [x] Streaming Chat Runtime (KV cache, sampling, `qpool` startup events)
- [x] Diffusion Pipeline (FlowMatch Euler, prompt encoder + denoiser + VAE decoder)
- [ ] Supported Pre-trained Models
- [x] [openai-community/gpt2](https://huggingface.co/openai-community/gpt2)
- [x] [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) (Gated model, requires request for access)
- [ ] [meta-llama/Llama-4-Scout-17B-16E](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E)
- [x] [black-forest-labs/FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B)
- [ ] [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it)
- [ ] [ibm-granite/granite-4.1-8b](https://huggingface.co/ibm-granite/granite-4.1-8b)
- [ ] [Qwen/Qwen3-Coder-Next](https://huggingface.co/Qwen/Qwen3-Coder-Next)
- [ ] [stabilityai/stable-diffusion-3-medium](https://huggingface.co/stabilityai/stable-diffusion-3-medium)
- [ ] [meituan-longcat/LongCat-AudioDiT-3.5B](https://huggingface.co/meituan-longcat/LongCat-AudioDiT-3.5B)
- [ ] [facebook/ijepa_vith16_1k](https://huggingface.co/facebook/ijepa_vith16_1k)
- [ ] [facebook/vjepa2-vitg-fpc64-256](https://huggingface.co/facebook/vjepa2-vitg-fpc64-256)
- [ ] Visual Node-Graph Architecture Builder
- [ ] ModelScope deep inspection tools
- [ ] Layer Surgery tools
- [ ] Hyperparameter Tuner
- [ ] Distributed Training
- [ ] Integrated Benchmarking Suite
- [ ] Deeply Integrated A.I. Assistant and Research Team
- [ ] Ergonomic WYSIWYG LaTeX Paper Editor
- [ ] Multi-User/Team Collaboration
## 🚀 Quick start
```bash
go install github.com/theapemachine/caramba@latest
caramba serve
```
The above command brings up just the HTTP API, which means you still have to bring up the data stores yourself.
Alternatively you could grab the `docker-compose.yml` from this repository to make this process much easier.
`caramba research ` lays out a project as a directory under version control:
```
research/project/my-ablation-study/
├── manifest/
│ ├── architecture/ # the architectures under comparison
│ └── operation/ # custom operations specific to this study
└── paper/ # write-up
```
Configuration lives in [cmd/asset/config.yml](cmd/asset/config.yml) and is loaded through [pkg/config](pkg/config). The resolver tries `--config`, then `./cmd/asset/config.yml`, `./config.yml`, `$HOME/.caramba/config.yml`, and finally the binary's embedded default. Data-store clients (`pkg/store/*`) read `store.qdrant`, `store.neo4j`, `store.elasticsearch`, and `store.deeplake` from that file (secrets may use `${...}` expansion); they do not call `os.Getenv` directly.
→ [Getting Started](./docs/getting-started.md)
## Building for a specific backend
```bash
# CPU — Go + AVX2/SSE2/NEON. Always available.
go build ./pkg/backend/compute/cpu/...
# CUDA — Linux, NVIDIA CUDA toolkit
CGO_ENABLED=1 go build -tags "cgo cuda" ./pkg/backend/device/cuda/...
# Metal — macOS, Xcode command-line tools (darwin + cgo selects Metal automatically)
go generate ./pkg/backend/device/metal
CGO_ENABLED=1 go build -tags cgo ./pkg/backend/device/metal/...
# XLA via PJRT — configure compute.xla in cmd/asset/config.yml first
go build -tags "cgo xla" ./pkg/backend/device/xla/...
```
```go
type Runner interface {
Execute(
ctx context.Context,
graph *ir.Graph,
targets []*ir.Node,
) (map[string]tensor.Tensor, error)
Location() tensor.Location
Close() error
}
```
Backend kernels upload values once into a resident tensor store and only download at real boundaries. The executor releases owned dependencies after their last graph consumer; the host arena reuses released spans. Host-staged dispatch is restricted to the host backend, so Metal, CUDA, and XLA paths cannot silently route through CPU slices.
→ [Compute Backends](./docs/compute.md)
## 💾 Repository layout
```
cmd/ Cobra CLI: serve, chat, image, research
asset/config.yml The single config source
pkg/
manifest/ YAML → IR compiler, registry, lowering
runtime/ Manifest runtime programs, state, ops, schedulers, graph bridge
backend/
compute/ Runner interface + cpu/, cuda/, metal/, xla/
api/ HTTP server
hub/ Hugging Face cache, Xet CAS
tokenizer/ ByteLevel BPE
model/ Weight binding, SafeTensors loader
notary/ Identity + provenance ledger
store/ S3, Elasticsearch, Neo4j, Qdrant, DeepLake
config/ Single config gateway
frontend/ Vite + React + Flume node editor
docs/ Long-form documentation
AGENTS.md Backend implementation contract — required reading for kernel work
```
---
## 🔬 Testing
Every code file has a `_test.go` mirror. Tests are GoConvey-style ("Given X, it should Y", nested). Backend kernels run parity tests against the scalar reference at `N ∈ {1, 7, 64, 1024, 8192}` with tight ULP bounds — the tolerance is a contract, not a knob.
```bash
go test ./...
CGO_ENABLED=1 go test -tags cgo ./pkg/backend/device/metal/...
CGO_ENABLED=1 go test -tags "cgo cuda" ./pkg/backend/device/cuda/...
go test -tags "cgo xla" ./pkg/backend/device/xla/...
```
---
## 📓 Documentation
| Document | What's inside |
|----------------------------------------------------------------|---------------------------------------------------------------------------------------------------|
| [Getting Started](./docs/getting-started.md) | Install, first chat, first study |
| [Architecture](./docs/architecture.md) | System design, IR, executor |
| [Manifest & Governance](./docs/manifest.md) | Manifest grammar, compiler pipeline, examples |
| [Compute Backends](./docs/compute.md) | CPU/SIMD, CUDA, Metal, XLA in depth |
| [Backend inventory](./docs/backend-inventory.md) | `device.Backend` methods ↔ `ir.RequiredOperationIDs()` |
| [CPU dispatch matrix](./docs/cpu-dispatch-matrix.md) | Per-domain scalar / AVX-512 / AVX2 / SSE2 / NEON registration |
| [Device backend matrix](./docs/device-backend-matrix.md) | Metal / CUDA / XLA kernel registrations, dtypes, required-op coverage |
| [Backend coverage matrix](./docs/backend-coverage.md) | Combined T1.2–T1.4 registration snapshot and R1 execution-target summary |
| [Backend compliance audit](./docs/backend-compliance-audit.md) | T1.6 machine checks: forbidden phrasing, cross-ISA calls, amd64 scalar tails, loose test epsilons |
| [Operations](./docs/operations.md) | Operation library, SIMD kernels, custom ops |
| [Frontend & Visualization](./docs/frontend.md) | Node editor, microscope tooling |
| [The Notary](./docs/notary.md) | Identity, ledger, custody model |
| [Agents](./docs/agents.md) | Conversational ingress, LLM providers |
| [AGENTS.md](./AGENTS.md) | Backend implementation contract for contributors |
---
## License
MIT