https://github.com/spcl/spada
Programming language and compiler for spatial dataflow architectures, such as Cerebras WSE.
https://github.com/spcl/spada
Last synced: 7 days ago
JSON representation
Programming language and compiler for spatial dataflow architectures, such as Cerebras WSE.
- Host: GitHub
- URL: https://github.com/spcl/spada
- Owner: spcl
- License: bsd-3-clause
- Created: 2024-02-12T12:26:53.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2026-04-24T09:02:20.000Z (4 months ago)
- Last Synced: 2026-07-12T19:33:14.284Z (27 days ago)
- Language: Python
- Homepage:
- Size: 1010 KB
- Stars: 5
- Watchers: 5
- Forks: 1
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Notice: NOTICE
Awesome Lists containing this project
README
# SPADA — A Spatial Dataflow Architecture Programming Language
SPADA is a programming language and compiler for spatial dataflow architectures such as the [Cerebras Wafer-Scale Engine](https://www.cerebras.net/). It provides precise control over data placement, communication streams, and asynchronous execution while abstracting architecture-specific routing details. SPADA also serves as a compiler intermediate representation (IR) for domain-specific languages; this repository includes a complete end-to-end compilation pipeline from the [GT4Py](https://github.com/GridTools/gt4py) stencil DSL (used in production weather forecasting at CSCS/MeteoSwiss) to Cerebras CSL.
Spatial dataflow architectures achieve exceptional throughput through disaggregated memory: each processing element (PE) holds only fast local SRAM, eliminating cache hierarchies and shared-memory contention. However, programming these architectures demands explicit orchestration of data movement over a circuit-switched network-on-chip (NoC), with limited concurrent communication channels and asynchronous, data-triggered task execution. SPADA addresses this by offering high-level constructs—`place`, `dataflow`, and `compute` blocks; `async`/`await`; `foreach` and `map` loops—alongside a formal dataflow semantics that defines routing correctness, data races, and deadlocks at compile time.
Key capabilities:
- **Explicit placement and dataflow**: Declare where data lives and how it moves between PEs.
- **Automatic routing assignment**: A checkerboard decomposition algorithm guarantees conflict-free channel allocation by construction, eliminating manual reasoning about hardware routing.
- **Multi-level compilation**: GT4Py stencils → Stencil IR → SPADA IR → Cerebras CSL, with automatic vectorization via Data Structure Descriptors (DSDs) and task fusion.
- **Compact code**: Hand-written SPADA kernels require 6–8× fewer lines than equivalent CSL; GT4Py stencils compile with up to 700× code reduction.
- **Near-ideal weak scaling**: Compiler-generated stencil kernels achieve >150 TFlop/s on the WSE-2 with near-ideal weak scaling across three orders of magnitude.
For full details, see the paper:
> Lukas Gianinazzi, Tal Ben-Nun, Torsten Hoefler. *SPADA: A Spatial Dataflow Architecture Programming Language.* arXiv:2511.09447, 2025.
---
## Getting Started
### Prerequisites
- Python ≥ 3.8
- [Cerebras SDK](https://sdk.cerebras.net/) (required to compile and run generated CSL code on WSE hardware; optional for compiler development)
### Installation
Clone the repository and install the package:
```bash
git clone https://github.com/glukas/spada.git
cd spada
pip install -e .
```
To install with development dependencies:
```bash
pip install -e ".[dev]"
```
### Compiling a SPADA Program
The `sptlc` command-line tool compiles a SPADA Spatial IR (`.sptl`) file to Cerebras CSL:
```bash
sptlc samples/benchmarks/laplacian_128_128_80.sptl output/ --param I=128 --param J=128
```
The first positional argument is the input `.sptl` file; the second is the output folder where CSL sources and `metadata.json` will be written.
Key options:
| Flag | Description |
|---|---|
| `--param NAME=VALUE` | Instantiate a compile-time parameter |
| `--offset-x N` / `--offset-y N` | PE grid offset on the fabric |
| `--generate-only` | Emit CSL source without invoking `cslc` |
| `--disable-benchmarking` | Omit cycle-counter instrumentation |
### Compiling from GT4Py
To compile a GT4Py stencil file to SPADA IR (`.spst` and `.sptl`):
```bash
python -m spada.cli.gt4py_to_spatial samples/stencils.py 128,128,80 output/ --function-name laplacian
```
Arguments in order: `input_file`, `domain_size` (comma-separated `x,y,z`), `output_dir`. Omitting `--function-name` compiles all stencils in the file.
The resulting `.sptl` file can then be passed to `sptlc`.
### Running on Hardware
After compiling with `cslc` (invoked automatically by `sptlc` unless `--generate-only` is set), run the kernel via the Cerebras `cs_python` launcher:
```bash
cs_python spada/runtime/runtime.py output/ in_field.npy
```
Alternatively, use the `Program` class directly from Python (must be run with `cs_python`):
```python
from spada.runtime.runtime import Program
import numpy as np
program = Program("output/")
in_field = np.random.rand(128, 128, 80).astype(np.float32)
outputs = program(in_field=in_field)
```
The runtime reads `metadata.json` generated by `sptlc` to determine the PE grid layout and I/O mapping automatically.
### Example Kernels
Sample SPADA programs are in `samples/`:
| Directory | Contents |
|---|---|
| `samples/stencils.py` | GT4Py stencil definitions (Laplacian, vertical advection, UVBKE, …) |
| `samples/advanced_stencils.py` | GT4Py definitions for horizontal diffusion kernels |
| `samples/benchmarks/` | Pre-compiled `.spst`/`.sptl` pairs for five kernels at five domain sizes |
| `samples/spatial/simple/` | Basic single-PE and streaming operations: `add`, `copy`, `forward_sum`, `backward_sum`, `mult_scalar`, `streaming_copy` |
| `samples/spatial/blas/` | Dense linear algebra: `axpy`, `matvec`, `gemv`, `gemv_twophase` |
| `samples/spatial/collectives/` | Reductions (`scalar`, `chain`, `tree`, `twophase` in 1D/2D) and broadcasts (`broadcast_1D`, `broadcast_2D`, and multicast variants) |
| `samples/spatial/stencils/` | Stencil examples: `laplacian` (high-level) and `laplacian_routed` (explicit routing) |
| `samples/spst/` | Stencil IR examples |
---
## SDK Version and WSE compatibility
The code has been tested for CSL SDK 1.4 and WSE-2.
## Testing
### Python Unit Tests
Python unit tests (no Cerebras SDK required) live under `tests/` and can be run with:
```bash
pytest tests/ --ignore=tests/csl_runtime
```
### CSL Runtime Tests (Singularity / Cerebras SDK)
End-to-end tests in `tests/csl_runtime/` compile and simulate SPADA programs using the Cerebras SDK and simulator. The Cerebras SDK ships as a Singularity Image File (`.sif`) and requires Singularity/Apptainer and an x86_64 Linux environment. Follow the Cerebras installation guide for full details: [Installation and Setup](https://sdk.cerebras.net/installation-guide).
**Linux or x86_64 VM setup**
1. Install Singularity/Apptainer as described in the SDK guide.
2. Download and extract the SDK in one step:
```bash
make -C tests/csl_runtime setup-sdk CSL_SDK_URL=
```
This saves the tarball to `tests/csl_runtime/cerebras-sdk.tar.gz` and extracts it to `tests/csl_runtime/cerebras-sdk/`. Both paths are gitignored. The extracted directory is used automatically by all subsequent `make` targets.
3. Install Python dependencies for the compiler:
```bash
python3 -m pip install -r requirements-ci.txt
```
4. Verify the toolchain:
```bash
make -C tests/csl_runtime check-sdk
```
5. Run the full CSL test suite:
```bash
make -C tests/csl_runtime test
```
**Run a single test:**
```bash
make -C tests/csl_runtime test-one TEST=test_add.sh
```
**Optional: run the SDK smoke test** against the `csl-extras-*` examples bundle:
```bash
make -C tests/csl_runtime smoke-sdk SDK_EXAMPLES_DIR=/path/to/csl-extras-*
```
If you have an SDK installed at a custom location, pass `CSL_SDK_DIR` to any target:
```bash
make -C tests/csl_runtime test CSL_SDK_DIR=/absolute/path/to/cs_sdk
```
**Apple Silicon macOS**
The Cerebras SDK documentation recommends running on Apple Silicon via an x86_64 Lima VM. A helper script handles everything — VM creation, SDK download, extraction, and test execution — in one command:
```bash
brew install lima qemu lima-additional-guestagents # one-time
tests/csl_runtime/run-in-lima.sh --sdk-url
```
This creates the Lima VM on first use (~5–10 min), downloads and extracts the SDK to `tests/csl_runtime/cerebras-sdk/`, installs Python dependencies inside the VM, and runs the full test suite.
If the SDK tarball is already downloaded or extracted, use `--sdk /path/to/cs_sdk` instead of `--sdk-url`.
Other modes:
```bash
# Run a single test
tests/csl_runtime/run-in-lima.sh --sdk --test test_add.sh
# Verify the SDK toolchain only
tests/csl_runtime/run-in-lima.sh --sdk --check
# Run the Cerebras SDK smoke test
tests/csl_runtime/run-in-lima.sh --sdk --smoke /path/to/csl-extras-*
# Drop into an interactive shell inside the VM
tests/csl_runtime/run-in-lima.sh --sdk --shell
```
The repository must reside under `$HOME` (Lima mounts the Mac home directory by default). The Lima configuration is in `tests/csl_runtime/lima-ubuntu-x86_64.yaml`.
**Cleanup** generated test artifacts:
```bash
make -C tests/csl_runtime clean # remove compiled output and .npy files
make -C tests/csl_runtime clean-sdk # also remove the downloaded SDK
```
---
## Getting Involved
Questions, discussions, and feedback are welcome via GitHub Issues:
- **Bug reports and feature requests**: [GitHub Issues](https://github.com/glukas/spada/issues)
---
## Contributing
Contributions are welcome. Please follow these steps:
1. **Fork** the repository and create a branch from `main`.
2. **Install** development dependencies: `pip install -e ".[dev]"`
3. **Write tests** for any new functionality. Tests live in `tests/` and are organized by subsystem (`stencil_ir/`, `spatial_ir/`, `placement/`, `gt4py/`, `csl_runtime/`).
4. **Format** your code with `black` and `isort`, and verify with `flake8`:
```bash
black spada tests
isort spada tests
flake8 spada tests
```
5. **Run tests**: see the [Testing](#testing) section for Python unit tests and CSL runtime tests.
6. **Open a pull request** against `main` with a clear description of the change and its motivation.
For significant changes (new language constructs, compiler passes, or architecture support), please open an issue first to discuss the design.
## Release
SPADA is released under BSD-3-Clause License, see [LICENSE](LICENSE) for details.
LLNL-CODE-2000963