https://github.com/alexjreid/pitboss
pitboss is a toy limit order book and matching engine
https://github.com/alexjreid/pitboss
experiment matching-engine order-book
Last synced: 9 days ago
JSON representation
pitboss is a toy limit order book and matching engine
- Host: GitHub
- URL: https://github.com/alexjreid/pitboss
- Owner: AlexJReid
- Created: 2026-05-27T05:30:10.000Z (13 days ago)
- Default Branch: main
- Last Pushed: 2026-05-27T12:02:08.000Z (12 days ago)
- Last Synced: 2026-05-27T13:25:39.907Z (12 days ago)
- Topics: experiment, matching-engine, order-book
- Language: C
- Homepage:
- Size: 4.09 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# pitboss
pitboss is a tiny deterministic matching engine in C.
It takes commands, writes them to a journal, emits events, and rebuilds the book
by replaying the journal.
Right now scope is narrow... one symbol, limit orders only, fixed capacities, and one
mutation owner for sequencing, persistence, matching, and book updates.
Commands are **input facts**, events are **engine decisions**, and the book is **derived state** rebuilt from the journal.
See [theory.md](./theory.md) for limit-order-book background,
[docs.md](./docs.md) for protocol and architecture details, and
[roadmap.md](./roadmap.md) for current status and planned work.
Some mechanical C/docs/sanity tests and benchmarks were written with non-human assistance. The design/layout/trade offs are mostly human, for better or worse.
I've taken some ideas from [monoblok](https://github.com/lexvicacom/monoblok).
## Why this exists
pitboss is not an exchange. It is a matching-engine core for looking at how
ordered systems sequence commands, produce events, persist decisions, and
rebuild state.
There is no FIX, auth, clearing, risk engine, or market data. My focus is onkeeping matching, persistence, replay, and recovery simple enough to reason about.
Some of the shape comes from a long-running interest in the ideas around event sourcing and [the LMAX architecture](https://martinfowler.com/articles/lmax.html), made famous in a classic 2011 write-up. Single-threaded in-memory business core, event sourcing, journal replay, and I/O pushed out to the edges. I have always found
that odd sounding approach interesting, and I have also seen a degree of
bandwagon jumping around it, a decade after it was a thing.
This project is not trying to build a photocopier version of a 2010
approach in 2026. In C, many of the Java workarounds are plainly nonsense: there is no
JVM object model to fight, no garbage collector to placate, and no reason to
copy ceremony from a different runtime. pitboss does not use a ring
buffer as its core architecture just because that was the famous part as blind
copying would miss the point. The choices here are simple: single-owner
mutation instead of lock-free core data structures, minimum allocation, no
hot-path allocation in matching, fixed pools when capacity matters, and
ordinary arrays and linked lists where they fit. Durability concerns still
matter: keep the mutation owner clear, make state replayable, keep I/O out of
the deterministic core, measure the hot path, and let data structures fit the
domain instead of the other way around.
## What it does
- Matches limit orders with price-time priority.
- Writes sequenced commands to a binary journal.
- Replays the journal to rebuild state.
- Saves snapshots as recovery shortcuts.
- Exposes a TCP gateway, monitor stream, and warm follower.
- Uses fixed pools and explicit capacity errors.
## Text protocol example
Clients send newline-delimited commands such as
`NEW_LIMIT order_id side price qty`.
Over TCP, an accepted command returns `+OK sequence event-count`, followed by
that many event lines. The sequence is the journal position; it is separate from
the order id.
Assume the book already has resting ask order `19`: sell `9` at price `101`,
and the next journal sequence is `19`.
```text
> NEW_LIMIT 41 S 200 10
< +OK 19 2
< ACCEPTED 19 41
< RESTING 19 41 S 200 10
> NEW_LIMIT 42 B 500 10
< +OK 20 3
< ACCEPTED 20 42
< TRADE 20 42 19 101 9
< TRADE 20 42 41 200 1
```
Order `41` is accepted at sequence `19` and rests because it does not cross.
Order `42` is a buy with limit `500`, so it can trade with asks priced `500` or
less. It consumes the cheaper resting ask first: order `19` at maker price
`101` for quantity `9`, then order `41` at maker price `200` for quantity `1`.
The buy is fully filled, so there is no `RESTING` event for order `42`.
## Build
```sh
cmake -S . -B build
cmake --build build
```
## Run
```sh
build/pitboss run input.txt
```
Use `-` for stdin:
```sh
printf 'NEW_LIMIT 1 B 100 10\nNEW_LIMIT 2 S 99 4\nCANCEL 1\n' | build/pitboss run -
```
The CLI appends binary command records to `pitboss.journal` by default before
applying them. Set `PITBOSS_JOURNAL=none` to disable journaling, or set it to a
different path.
Journal writes use fixed 64-byte records and a 64 KiB buffer. The TCP gateway
acks accepted commands only after their journal buffer flushes; the default
group-commit timer is 1 ms. Set `PITBOSS_JOURNAL_FLUSH_MS=N` to change it, and
`PITBOSS_JOURNAL_FSYNC=1` to fsync each flushed buffer.
Replay a sequenced journal and print the same events again:
```sh
build/pitboss replay pitboss.journal
```
The replay output format is described in [docs.md](./docs.md#replay-output).
Dump the derived book state after replaying a journal:
```sh
build/pitboss dump pitboss.journal
```
`build/pitboss dump` without a path reads `PITBOSS_JOURNAL` when it is set,
otherwise `pitboss.journal`.
Create a binary checkpoint of the current derived book, then recover from that
checkpoint plus any later journal records:
```sh
build/pitboss checkpoint pitboss.journal pitboss.snap
build/pitboss recover pitboss.journal pitboss.snap
```
When `run` or `listen` append to an existing binary journal, pitboss first
replays that journal silently so sequence numbers and the in-memory book continue
from the recovered state.
## TCP Gateway
The TCP listener is an I/O shell around the deterministic core. Socket framing,
parse work, and stateless validation can run outside the sequencer; sequence
assignment, journal append, matching, book mutation, monitor fanout, and
replication fanout stay ordered on the libuv loop.
```mermaid
flowchart LR
client["client commands"]
gate["gate / validation"]
sequencer["sequencer"]
journal["journal"]
matcher["matcher"]
events["events"]
book["book state"]
monitors["monitor clients"]
replicas["warm followers"]
client --> gate --> sequencer --> journal --> matcher --> events --> book
events --> monitors
journal --> replicas
```
```sh
build/pitboss listen 127.0.0.1 17077
build/pitboss listen 127.0.0.1 17077 17078
build/pitboss listen 127.0.0.1 17077 17078 17079
scripts/pitboss-client.py 127.0.0.1:17077
scripts/pitboss-client.py 127.0.0.1:17077 -c 'NEW_LIMIT 1 B 100 10'
scripts/start-replication-pair.sh
```
The TCP listener handles `SIGINT` and `SIGTERM` as graceful shutdown requests:
it closes the listeners and active sessions, then runs the normal journal close
path. Set `PITBOSS_JOURNAL_FSYNC=1` when flushed journal buffers should be
fsynced.
When the optional monitor port is present, monitor clients receive
`PITBOSS MONITOR 1`, a current book snapshot, then a read-only live event
stream.
When the optional replication port is present, warm followers can connect and
tail the primary's sequenced binary journal stream to maintain their own journal.
```sh
PITBOSS_JOURNAL=follower.journal build/pitboss follow 127.0.0.1 17079
```
The follower recovers its local journal, sends a byte/sequence cursor, then
appends and applies primary records after that point. Client ingress remains
single-primary.
For operator-driven promotion of follower to leader, inspect the follower journal and compare it with
the old primary monitor sequence. See `scripts/start-replication-pair.sh` and the temp utility scripts it generates.
## Test
```sh
ctest --test-dir build
scripts/listener-smoke.sh
scripts/monitor-smoke.sh
scripts/replication-smoke.sh
```
Useful manual checks:
```sh
build/pitboss bench 100000
build/pitboss bench-levels 1000000
build/pitboss bench-workload matcher-only --commands 3000000 --active-orders 1000 --price-slots 750 --cross-rate 6
build/pitboss bench-workload journaled --commands 3000000 --active-orders 1000 --price-slots 750 --cross-rate 6
scripts/replication-bench.sh catchup 10000
scripts/replication-bench.sh live 10000
scripts/replication-bench.sh live-bulk 100000
```
The Python client accepts multiple command-gateway endpoints and tries them in
order. If the current connection drops, it reconnects to the next endpoint:
```sh
scripts/pitboss-client.py 127.0.0.1:17077 127.0.0.1:18077
scripts/pitboss-client.py 127.0.0.1:17077 127.0.0.1:18077 -f commands.txt --strict
```
`scripts/start-replication-pair.sh` starts a local leader with command, monitor,
and replication ports plus one follower. It writes journals, logs, and PID files
under a temp run directory. It also generates `env.sh` plus helper scripts for
client, monitor, promote, stop, journal info, and log tailing.
`bench-workload matcher-only` measures the in-process matcher on a deterministic
active-book workload. `bench-workload journaled` includes append-only journal
writes on the same workload. Both report throughput, command mix, final active
order count, and p50/p90/p99/worst latency.
Example output from one M2 MacBook Air (2022) run, Release build. Treat these as
orientation numbers, not a benchmark claim.
| Surface | Command | Workload | Throughput | Latency |
| --- | --- | ---: | ---: | ---: |
| Matcher only | `bench-workload matcher-only` | 3,000,000 commands | 10.4M cmd/s | p50 83ns, p99 167ns |
| Journaled matcher | `bench-workload journaled` | 300,000 commands | 497k cmd/s | p50 1.5us, p99 4.9us |
| Replication catch-up, journaled follower | `replication-bench.sh catchup` | 10,000 records | 203k records/s | batch catch-up |
| Live tail, client-paced | `replication-bench.sh live` | 10,000 records | 9.3k records/s | one command at a time |
| Live tail, bounded bulk | `replication-bench.sh live-bulk` | 10,000 records | 13.8k records/s | 256-command window |
`replication-bench.sh live` sends one command at a time and waits for the client
response. It is a conservative end-to-end loopback check, not a maximum fanout
number.
## Commands
Protocol, matching semantics, architecture notes, test commands, and current
limitations live in [docs.md](./docs.md#commands).