https://github.com/root-11/intro-book-python
https://github.com/root-11/intro-book-python
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/root-11/intro-book-python
- Owner: root-11
- Created: 2026-05-09T20:35:00.000Z (2 months ago)
- Default Branch: gh-pages
- Last Pushed: 2026-06-01T19:03:51.000Z (about 2 months ago)
- Last Synced: 2026-06-01T20:23:48.369Z (about 2 months ago)
- Language: HTML
- Size: 5.06 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# An Introduction to Programming, using ECS & EBP in Python
**Read it online: [Codeberg](https://root-11.codeberg.page/intro-book-python/) · [GitHub Pages](https://root-11.github.io/intro-book-python/)**
A book that teaches programming from first principles of data-oriented design, entity-component-systems (ECS), and existence-based processing (EBP). Uses Python and `numpy` as the only languages.
The through-line is a small ecosystem simulator built in stages from one hundred wandering creatures to a hundred million streamed ones. Forty-four sections; two-to-three pages of prose plus four-to-twelve compounding exercises each.
This is the **Python edition** - a sister volume to the [Rust edition](https://root-11.codeberg.page/intro-book/) of the same book. Same forty-four sections, same DAG, same simulator. The variation is per-chapter commentary on what Python's defaults push the reader into, and why ECS and EBP win even in a slow language.
Found something wrong, unclear, or worth adding? [Open an issue.](https://codeberg.org/root-11/intro-book-python/issues)
_written by [Bjorn Madsen](mailto:dr.bjorn.madsen@gmail.com)_
_updated: 2026-06-07_
> **Read online:** [Codeberg](https://root-11.codeberg.page/intro-book-python/) · [GitHub Pages](https://root-11.github.io/intro-book-python/)
>
> **Clone source** (the public default branch is the rendered book; the runnable code lives on `main`): `git clone --branch main https://codeberg.org/root-11/intro-book-python.git` · `git clone --branch main https://github.com/root-11/intro-book-python.git`
>
> **Issues:** [Codeberg](https://codeberg.org/root-11/intro-book-python/issues) · [GitHub](https://github.com/root-11/intro-book-python/issues)
This book teaches programming from first principles of data-oriented design, entity-component-systems (ECS), and existence-based processing (EBP). It uses Python and `numpy` as the only languages.
*EBP* is this book's shorthand. The spelled-out term - *existence-based processing* - is Richard Fabian's, from [*Data-Oriented Design*](https://www.dataorienteddesign.com/dodbook/); §17 builds it from the simulator. An acronym index will not list "EBP" because the source literature spells the term out rather than abbreviating it.
The book is structured around forty-three concepts ([the DAG](https://root-11.codeberg.page/intro-book-python/concepts/dag.html)) and their canonical wording ([the glossary](https://root-11.codeberg.page/intro-book-python/concepts/glossary.html)). Sections are short - two to three pages of prose followed by four to twelve compounding exercises. Concepts are *named* only after they are *built*: every section earns its vocabulary through working code, not the other way around.
The through-line is a small ecosystem simulator built in stages from one hundred wandering creatures to a hundred million streamed ones. The simulator's specification is at [`code/sim/SPEC`](https://root-11.codeberg.page/intro-book-python/code/sim/SPEC.html).
This is the **Python edition** - a sister volume to the Rust edition of the same book. Same forty-four sections, same DAG, same simulator. The variation is per-chapter commentary on what Python's defaults push the reader into, and why ECS and EBP win even in a slow language. The thesis the edition carries: **ECS and EBP beat OOP because they process more efficiently (operations grouped over arrays), they extend more cleanly (data-oriented composition over class graphs), and they have smaller memory footprint (typed columns over object graphs).**
What carries this edition is the **evidence**. Every load-bearing claim is backed by a measurement the reader can reproduce on their own laptop in under a minute. The exhibits live in [`code/measurement/`](https://github.com/root-11/intro-book-python/tree/main/code/measurement) and run via `uv run code/measurement/.py`.
This is a work in progress. Section ordering is by the DAG; reading order can be linear (front to back) or by following the cross-links wherever they lead.
## Who this book is for
You used Python last week. You wrote a class, put instances in a list, iterated over them. Your code worked, but it was slower than you expected, and you have started wondering whether the standard idioms are the bottleneck.
This book is for people who want to find out. The premise is that they are - and that the architecture this book teaches is what Python is fast in, when Python is fast at all.
Many online books include a playground that runs the code in your browser. This one does not, on purpose: the measurements only mean something when they come from *your* hardware.
## Background
You should be comfortable with high-school algebra and a command line - running a command, changing directories, reading error messages without panic. A laptop with internet is enough; the book uses Python 3.11+, `numpy`, and `uv` for environment management. Everything else is standard library.
You do *not* need prior expertise in numerics, parallel computing, or game development. The book teaches numpy and the simulator together; the language is a vehicle, not the subject.
## A first taste
Before any vocabulary is named, here is what an ECS world looks like in fifteen lines of Python. One hundred creatures, each with a position and a velocity, moving for thirty ticks of simulated time. No classes, no instances, no method calls - four `numpy` arrays indexed in lockstep, and a function (the per-tick update) that advances every creature in one stride.
```python
import numpy as np
n = 100
x = np.arange(n, dtype=np.float32) * 0.1
y = np.sin(np.arange(n, dtype=np.float32))
vx = ((np.arange(n) * 7) % 11).astype(np.float32) * 0.01 - 0.05
vy = ((np.arange(n) * 13) % 7).astype(np.float32) * 0.01 - 0.03
for tick in range(30):
x += vx
y += vy
if tick % 10 == 0:
print(f"tick {tick}: creature 17 at ({x[17]:.2f}, {y[17]:.2f})")
```
Run it locally. Three lines print, the script stops. That is the entire shape of what the rest of the book grows: tables (the four arrays), a tick (the outer loop), a system (the per-tick update). Everything that follows is the discipline that lets this same shape carry a hundred million creatures without falling apart.
The familiar Python shape - a `Creature` class, a list of instances, a `step()` method - works at this size too. It stops working at a million, and the reason is in [§2](book/trunk/02_numbers_and_how_they_fit.md): an order of magnitude more memory per creature, an order of magnitude slower per tick. The book teaches the layout that survives the next zero.
## Running the code
Python has no equivalent of the Rust Playground - there is no browser-hosted runner that reproduces the numbers a chapter quotes. Every measurement and exhibit in this book runs locally, using [`uv`](https://docs.astral.sh/uv/) to manage the Python toolchain and environment. To run anything, you will want a clone of the book's repo:
```sh
git clone https://codeberg.org/root-11/intro-book-python.git
cd intro-book-python
uv run code/measurement/cache_cliffs.py
```
Each `code/measurement/.py` file is one exercise group, runnable in isolation. The numbers it prints are *yours* - they come from your hardware. The exercise asks "how fast does *your* machine run this?", and that question only has a real answer locally.
From the simulator chapters onward (§11+), the exercises stop being self-contained scripts. They build the through-line: a Python program that grows from one hundred wandering creatures to a hundred million streamed ones. That program holds state between runs, which is what `uv run` and the project layout buy you.
## The companion edition
If you already know Python well and want compile-time enforcement of the discipline this book teaches by convention, the [Rust edition](https://root-11.codeberg.page/intro-book/) covers the same forty-four sections in Rust. The architecture is identical; the language differs. Many readers find that watching the borrow checker enforce in Rust what this edition asks for as discipline is a useful calibration in the other direction too.
# Nomenclature
Quick reference for symbols, notation, and abbreviations the book uses. Concept *definitions* live in the [glossary](https://root-11.codeberg.page/intro-book-python/concepts/glossary.html); this page covers the shorthand only.
## Symbols
| Symbol | Meaning |
|---|---|
| §N | Section number - e.g., §5 refers to section 5. |
| → | Leads to / becomes / transitions to. Appears in section titles (e.g., §29 "10K → 1M") and prose. |
| `[!NOTE]` / `[!TIP]` / `[!WARNING]` | Callout box - content the reader should pay particular attention to. |
## Text formatting
| Form | Meaning |
|---|---|
| `monospace` | Code: types, variable names, function names, file paths. |
| *italic* | First definition of a term, or emphasis. |
| **bold** | A term being highlighted as load-bearing in the current paragraph. |
| `# anti-pattern: bad!` | A code comment that flags the snippet as something the chapter is arguing *against*. The label travels with the code if a reader copy-pastes. |
## Variables you will see across chapters
| Variable | Meaning |
|---|---|
| `i`, `j` | Index into a column. `i` is the index of the row currently under discussion. |
| `t` or `tick` | Tick number - the simulator's step counter. |
| `id` | Stable entity identifier (a small unsigned integer; usually `np.uint32`). |
| `gen` | Generation counter, paired with a slot index to detect stale references (§10). |
| `pos_x`, `pos_y` | Position columns of a creature (`np.float32`). |
| `vel_x`, `vel_y` | Velocity columns of a creature (`np.float32`). |
| `to_remove`, `to_insert` | Buffers of pending mutations applied at end-of-tick (§22). |
| `n_active` | Length of the live prefix of a fixed-capacity column (§21, §24). |
## Python types and their numpy counterparts
This book uses numpy's typed dtypes for hot data. The mapping the reader will see most often:
| Python | numpy | size | range |
|---|---|---|---|
| `int` (CPython, ≤ 2³⁰) | - | 28 bytes | unbounded |
| - | `np.int8` | 1 byte | -128 to 127 |
| - | `np.uint8` | 1 byte | 0 to 255 |
| - | `np.int32` | 4 bytes | ±2³¹ |
| - | `np.uint32` | 4 bytes | 0 to 2³² |
| - | `np.int64` | 8 bytes | ±2⁶³ |
| `float` (CPython) | `np.float64` | 8 bytes (CPython has 24-byte object overhead) | ~15 decimal digits |
| - | `np.float32` | 4 bytes | ~7 decimal digits |
| - | `np.bool_` | 1 byte (in arrays) | True / False |
## Conventions for code blocks
| Form | Convention |
|---|---|
| Plain triple-backtick `python` | A snippet to read; not necessarily complete. |
| Snippet with `# anti-pattern: bad!` first line | A snippet shown as the *wrong* way; the chapter is about the right way. |
| `uv run code/measurement/.py` | A measurement exhibit the reader can run on their machine. The numbers in the chapter were measured the same way. |
# 1 - The machine model
Most explanations of "how a computer works" use a diagram with a CPU and a single big block called *memory*. The diagram is wrong. Memory is many things at different speeds, and which one your data sits in decides whether your program is fast or slow.
Inside the CPU there is **L1 cache** - small, sometimes only 32 KB per core, but a read from it costs about one nanosecond. Around it sits **L2** - a few hundred KB, around 3-4 ns. Then **L3** - measured in megabytes, around 10 ns. Outside the CPU sits **main memory (RAM)** - gigabytes, around 100 ns per read. The numbers vary by chip; the *ratios* are stable. L1 is roughly a hundred times faster than RAM.
When your code reads `arr[17]`, the CPU does not pull just byte 17. It pulls a whole 64-byte chunk - a *cache line* - and keeps that line in L1. The next read of `arr[18]` is then almost free. Reading sequentially is fast because every line that gets loaded is mostly used before it gets evicted. Reading at random is slow because every read costs a fresh trip to RAM.
A pointer is an address in memory. Following one is one memory read at an address the CPU does not get to predict. If the address is in cache, the read is fast; if not, you wait the full ~100 ns. A program with many objects and many pointers between them is a program with many of those waits.
## Why you have not had to think about this
If you used Python last week, none of the above came up. The interpreter ran your code, the operating system handed it memory, and *it worked*. You felt no cliff at 100 KB or 100 MB. You wrote a `for` loop, the loop ran, and the cost per element was whatever it was.
That experience is real, and it is hiding the machine from you. The cost of one iteration of a Python `for` loop - `PyObject_Add`, the refcount increment, the `PyLong` boxing, the bytecode dispatch - is around 5 nanoseconds per element on this machine. That number is *higher* than an L3 cache miss. So when you iterate over a Python list, the cache hierarchy is invisible to you: you spend so long in the interpreter on every step that whether the next byte was in L1 or had to come from RAM is rounding error.
This is the missing piece of the machine model in Python. The hierarchy is still there; the bottleneck just moved. To *see* the machine, you have to look in places where the interpreter dispatch isn't dominating. Two such places, both measurable on your laptop:
**1. Sum a million int64s, three ways.** [`code/measurement/cache_cliffs.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/cache_cliffs.py) walks N from 10K to 100M and times: `sum(lst)` on a Python list, `arr.sum()` on a contiguous numpy array, and `arr[idx].sum()` where `idx` is a shuffled permutation. On this machine:
| N | Python list | numpy seq | numpy gather | gather/seq |
|------------:|------------:|----------:|-------------:|-----------:|
| 10,000 | 4.85 ns | 0.65 ns | 3.07 ns | 4.7× |
| 100,000 | 4.60 ns | 0.37 ns | 2.01 ns | 5.4× |
| 1,000,000 | 4.60 ns | 0.21 ns | 3.53 ns | 17.0× |
| 10,000,000 | 4.62 ns | 0.15 ns | 10.06 ns | 66.0× |
| 100,000,000 | 4.60 ns | 0.15 ns | 11.72 ns | 80.0× |
Read the columns (3-run medians; the sub-nanosecond seq column is noisy run-to-run, the gather column and the RAM ratios are the stable claims). The Python list is **flat at ~4.6 ns/element across five orders of magnitude**. From inside the interpreter the cache hierarchy does not exist. The numpy sequential column is 25-30× faster and reveals the bandwidth - the inner loop is C, the bytes are typed, the prefetcher works. The numpy gather column is the same data accessed in a shuffled order; while it fits in cache the gather penalty is small (~5-17×), and once the working set spills L3 (between 1M and 10M) the per-element cost climbs sharply, reaching **80×** the sequential cost at 100M. That ratio is the L1-to-RAM cost gap on this machine, measured.
**2. Take an exception once vs a million times.** [`code/measurement/try_except.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/try_except.py) compares `try/except ZeroDivisionError` against an explicit `if value != 0` check, across hit rates from 0.0001% to 99.9999%. At 50/50 the `try/except` form is 4× slower; at 99.9999% (almost no exceptions raised) the `try/except` form is *faster* than the `if`. The difference is the CPU's branch predictor: a taken branch with high frequency is essentially free; a mispredicted one costs ~10-20 cycles. The lesson is not "use try/except" or "use if" - it is that constant factors are rate-dependent, and even Python inherits this.
**3. Constant factors leak through.** [`code/measurement/string_methods.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/string_methods.py) compares `%`-format, f-strings, and `.format` for the same output. On this machine `%`-format is ~20% faster than f-strings, which are ~5% faster than `.format`. None of this matters in a one-off log line. All of it matters in a tight loop. The "modern idiomatic" choice is not automatically the cheap choice.
## What this chapter is asking you to do
The dominant fact about modern CPUs is that arithmetic is virtually free; the cost is *getting the data to the arithmetic*. A program that respects this is fast. A program that ignores it can be a hundred times slower than a program that does the same work, with the same number of additions, in a layout the cache likes.
In Python this fact wears a disguise: the interpreter is so slow that the machine appears to have no cliff. The disguise comes off the moment you leave pure Python - and almost everything this book teaches involves leaving pure Python for typed contiguous columns where the cliff is right where it always was.
This is also what makes "complexity class" misleading on its own. An O(N log N) algorithm that hits the cache hard can outrun a "faster" O(N) algorithm that scatters reads across RAM. Big-O describes how cost grows with N; layout describes the constant factor that gets multiplied in. At the scales this book targets, the constant factor often wins.
## Exercises
These exercises are calibrations. Run them on your machine and write the numbers down - the rest of the book references them.
1. **Look up your cache sizes.** On Linux, `lscpu | grep -i cache` lists L1d, L1i, L2, L3 per core. (On macOS: `sysctl -a | grep cache`.) Write them down. These are the budgets [§27](#27---working-set-vs-cache) will hold you to later.
2. **Run the cache-cliffs exhibit.** `uv run code/measurement/cache_cliffs.py`. Read the output. Note the size at which the numpy gather column starts climbing - that is where you spilled out of L1. Note where it climbs again - L2, L3.
3. **Confirm the interpreter mask.** Modify the exhibit to print `arr.tolist()` sum at every size step alongside the existing measurements. Confirm that the Python list cost is still flat - the cliffs do not appear, even though the data is the same.
4. **Run the try/except exhibit.** `uv run code/measurement/try_except.py`. Note the cross-over: at what hit-rate does `try/except` become faster than `if`? On most machines it lands above 99%.
5. **Run the string-format exhibit.** `uv run code/measurement/string_methods.py`. Note the ranking on your machine. The order can shift across CPython versions - measure, do not memorise.
6. **A linked list of pointers.** Build a chain of 1,000,000 nodes as `class Node: __slots__ = ("value", "next")`, then sum `value` by walking `.next` from the head. Compare against the same sum on a numpy `int64` array of the same length. The ratio you see is roughly the L1-to-RAM ratio for *one* level of indirection in Python - note that this ratio compounds when objects nest deeper.
7. *(stretch)* **Read your `lscpu` output to your benchmarks.** With your cache sizes from exercise 1 and your timings from exercise 2, identify which level of cache each step in the gather column is leaving. The transitions are not always clean - annotate where they are noisy.
> [!NOTE]
> Numbers in this chapter were measured on this author's machine. The shape - flat Python list, staircase numpy, widening gather/seq ratio - is robust across hardware. The exact ratios shift with CPU generation: older or smaller chips (Raspberry Pi 4, 2012-era Intel) show a graded staircase across L1/L2/L3, while modern desktop chips often show one big cliff at the L3-to-RAM boundary. Measure on your own machine; reproduce shapes, not specific numbers.
Reference notes for these exercises in [01_the_machine_model_solutions.md](https://root-11.codeberg.page/intro-book-python/trunk/01_the_machine_model_solutions.html).
## What's next
The cache sizes you wrote down in exercise 1 and the cliffs you found in exercise 2 are the constants behind the whole book. [§2 - Numbers and how they fit](#2---numbers-and-how-they-fit) takes the next step: how big is each unit of data, and how many fit in a cache line?
# 2 - Numbers and how they fit
A cache line is 64 bytes on x86 and most ARM chips - the unit of memory the CPU loads at a time. (A few designs differ: some Apple Silicon cache levels use 128; §33 has the details.) This book assumes 64 throughout. Everything you do with data is, in part, a question of how many things fit in one cache line.
## What an `int` actually costs
You wrote `x = 1` last week and that was the end of the question. What sat in memory was a `PyLong` object: a header, a refcount, a length, and one or more 32-bit "digit" limbs holding the value. The minimum size, even for `0`, is **28 bytes** - and that covers every value below 2³⁰ (about a billion). Past that it grows four bytes per limb, one limb per 30 bits (≈ nine decimal digits), not four bytes per digit. From [`code/measurement/number_footprint.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/number_footprint.py) on this machine:
```
int 0 28 bytes
int 1 28 bytes
int 256 (last interned) 28 bytes
int 257 28 bytes
int 1_000 28 bytes
int 2**31 32 bytes
int 2**63 36 bytes
int 2**127 44 bytes
float 0.0 24 bytes
float 3.14 24 bytes
float 1e300 24 bytes
```
A `PyFloat` is 24 bytes, fixed. A `PyLong` is at least 28 bytes and grows with magnitude. A `bool` is also a `PyLong`. A `complex` is 32 bytes. The header alone is bigger than the value in every case.
This is the first part of the chapter's question. *Picking the narrowest type that holds your range* - the discipline that defines whether a cache line packs 8 things or 64 things - does not exist in pure Python. There is no `uint8`. There is no `int32`. Every Python `int` is the same costly object regardless of whether it holds the value `0` or `2**63`. You cannot trade range for cache lines, because you cannot pick the range.
> [!NOTE]
> CPython caches small integers in `[-5, 256]` as singletons (the *small-int cache*). A list of zeros does not allocate a million `PyLong(0)` objects - it allocates a million pointers, all to the same one. Once the values escape that range, every value is a fresh allocation. Confirm this with `id(0) == id(0)` (true) versus `id(257) == id(257)` (sometimes true, sometimes not, depending on the parser's caching of literal constants in the same compilation unit - but never reliable). Treat the small-int cache as a CPython implementation detail you cannot lean on. Terry Jan Reedy, a CPython core developer, [confirmed](https://discuss.python.org/t/there-is-a-better-way-a-free-python-book-on-ecs-ebp/107504/7) that 3.15.0b1 and newer raise that upper limit from 256 to 1024 - the boundary moved while this book was being written. The durable claim is the mechanism, not the number.
## What numpy gives you back
`numpy` makes the width budget exist again. `np.int8` is one byte, range -128 to 127. `np.int16` is two bytes, `np.int32` is four, `np.int64` is eight. `np.float32` is four bytes (~7 decimal digits of precision); `np.float64` is eight (~15 digits). The signed/unsigned and integer/float variants compose freely.
A `np.zeros(N, dtype=np.uint8)` is N bytes - flat, contiguous, no per-element header. A cache line packs **64** of them. A `np.zeros(N, dtype=np.int64)` is 8N bytes; one cache line packs **8**. Walk the whole array and the int64 version pulls in 8× as many cache lines as the uint8 version: the same element count, eight times the bytes. The width budget is back.
Same exhibit, the data column tells the story at N=1,000,000:
| layout | data size | sum (ms) |
|------------------------------------|----------:|---------:|
| Python list of large ints | 38.25 MB | 2.56 |
| Python list of floats | 38.38 MB | 4.27 |
| numpy int8 | 0.95 MB | 0.28 |
| numpy int16 | 1.91 MB | 0.34 |
| numpy int32 | 3.81 MB | 0.45 |
| numpy int64 | 7.63 MB | 0.42 |
| numpy float32 | 3.81 MB | 0.22 |
| numpy float64 | 7.63 MB | 0.36 |
The Python-list-vs-numpy ratio at this scale: **40× more bytes** in the list compared to numpy int8, **20×** vs int16, **10×** vs int32, **5×** vs int64. Choosing the narrowest numpy width that holds your range gives you up to 8× *additional* shrink on top of the list-to-numpy step. Sum times collapse from milliseconds to fractions of a millisecond - two orders of magnitude.
Pick the narrowest type that holds your range, and write down why. A 52-card deck's `suits` need 4 values, `ranks` need 13, `locations` need maybe 8 - all fit in `np.uint8`. A creature's `pos` needs about ten kilometres of grid resolved to centimetre precision; that fits in `np.float32`. A timestamp in microseconds for a year-long simulation needs something like 3×10¹³, which does not fit in `np.uint32` (4×10⁹) but fits comfortably in `np.uint64`. Choose, and write the choice down.
## Floats are not real numbers
They look like real numbers but are not. There are only about 4 billion `float32` values; there are only about 18 quintillion `float64` values; that is finite. Operations have edges: `1.0 / 0.0 = inf`, `0.0 / 0.0 = nan`, and `nan != nan` - yes, equality is broken on purpose for `nan`, because there is no reasonable answer. But `==` is also unreliable for *ordinary* floats: `0.1 + 0.2 == 0.3` is `False`, because `0.1` and `0.2` cannot be represented exactly in binary and the rounding error happens to land just past `0.3`. This is why `math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0)` exists - it is the standard library's acknowledgement that `==` is the wrong tool for floats and that comparing them needs a tolerance you choose deliberately. Subtracting two nearly equal floats loses most of their precision (this is *catastrophic cancellation*). Adding a tiny float to a large one quietly drops the tiny one (this is *absorption*). None of this is a problem if you know it is there; all of it is a problem if you assume floats are mathematics.
[`code/measurement/sums.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/sums.py) demonstrates the consequences across five pathological datasets - random balanced, large-plus-many-small, alternating signs, tiny increments, and arrays containing NaNs - using six summation algorithms (`sum`, `math.fsum`, Kahan, Neumaier, pairwise, decimal reference). Run it; read the discrepancies. The same input data summed in different orders gives different answers, and the "naive" answer is sometimes off by orders of magnitude. The fix is not "use float64 instead of float32" - it is *picking a summation algorithm aware of the data shape*. `math.fsum` and Neumaier are usually the right defaults for a single-pass sum where you cannot bound the input.
Most of this book uses `np.uint8`, `np.uint16`, `np.uint32`, `np.float32`, and `np.uint64` for time. `int*` and `float64` appear when the range or precision genuinely demands it. The choice is documented at every column declaration.
## Exercises
1. **Per-value cost.** Print `sys.getsizeof(0)`, `sys.getsizeof(2**31)`, `sys.getsizeof(2**127)`, `sys.getsizeof(0.0)`, `sys.getsizeof(True)`. Confirm that even a `bool` costs 28 bytes (`bool` is a subclass of `int`). Now print `np.array([0, 2**31, 0], dtype=np.int64).nbytes`. Three int64s = 24 bytes total, no headers, no per-value pointers.
2. **Cache-line packing.** For each numpy dtype - `int8`, `int16`, `int32`, `int64`, `float32`, `float64` - compute how many fit in a 64-byte cache line. A `np.array(_, dtype=np.int32)` of 16 elements is exactly one line; a `np.array(_, dtype=np.float64)` of 8 elements is exactly one line.
3. **Width and speed.** Sum a `np.ones(100_000_000, dtype=np.int8)`, then a `np.ones(100_000_000, dtype=np.int64)`. The ratio in time should be smaller than the ratio in bytes (8×) because compute is not the bottleneck - memory bandwidth is. Note also that the int8 sum overflows; this is a hint about why the book picks widths *with the maximum value in mind*.
4. **Float weirdness.** Compute `0.0 / 0.0`, `1.0 / 0.0`, `(-1.0) ** 0.5`, `math.sqrt(-1.0)`. Print them. Then `nan = float("nan"); assert nan != nan` - confirm it does not raise.
5. **`==` is the wrong tool.** Print `0.1 + 0.2 == 0.3`. Observe `False`. Print `0.1 + 0.2` to see the rounding error: `0.30000000000000004`. Now use `math.isclose(0.1 + 0.2, 0.3)` and observe `True`. Read [the `math.isclose` docs](https://docs.python.org/3/library/math.html#math.isclose) - note that the default `rel_tol=1e-9` is a *choice* you should be making explicitly when the problem demands a tighter or looser tolerance. The standard library has `isclose` because the language designers know `==` is unreliable here; lean on it.
6. **Catastrophic cancellation.** Compute `np.float32(1e10) - (np.float32(1e10) - np.float32(1.0))`. The result should be `1.0`; on `float32` it usually is not. Repeat with `np.float64` and observe it gets closer (but not always exactly `1.0`).
7. **Run the summation exhibit.** `uv run code/measurement/sums.py`. Read the discrepancies between the algorithms across the five datasets. Note the dataset where the spread is largest. That dataset is the one that decides which summation routine you should reach for in production.
8. **Choose a width.** For each of these columns, write down the dtype you would pick and why: a creature's age in ticks at 30 Hz over a year-long simulation; a card's suit; the pixel count of a 4K screen; the user id in a system with up to 100 million users; an audio sample value in 16-bit PCM.
9. *(stretch)* **The `eps` of a float.** `np.finfo(np.float32).eps` is the smallest `x` such that `1.0 + x != 1.0` in float32. Compute the value, then compute `np.float32(1.0) + np.float32(0.5) * np.finfo(np.float32).eps` - is the result `1.0` or `1.0 + eps/2`? What does this say about a sum of small numbers added one at a time to a large running total?
## What's next
[§3 - The `np.ndarray` is a table](#3---the-npndarray-is-a-table) takes the next step: now that you know how big the elements are, what does an `np.array` *do* with them, and what shape does the rest of the book expect them to be in?
# 3 - The `np.ndarray` is a table
A `list` in Python is a header object on the heap that stores three things: a length, a capacity (over-allocated by a small fraction), and a pointer to a contiguous run of `PyObject*` *pointers*. That last word is the lesson. The `list` does not contain your integers; it contains pointers to integer *objects*, each allocated separately on the heap. `lst[i]` reads a pointer from the contiguous run, then dereferences it to find the actual `PyLong` (28 bytes per int, 24 per float) somewhere else in memory.
If you used Python last week, this is the container you reached for, and it is the right shape for *some* problems. It is also the wrong shape for almost everything the trunk of this book teaches, which is "process all the rows of a table." A `list` of N rows-as-tuples is one big jump table sitting in front of N+10N small objects scattered across the heap. Walking it is pointer-chasing, not sequential reading.
A `numpy` array - `np.array(..., dtype=...)` - is the same three-things-on-the-heap shape, but the contiguous run holds *values*, not pointers. Ten million int64s in a numpy array is 80 MB of contiguous bytes; ten million ints in a list is 280 MB of `PyLong` objects plus 80 MB of pointers, scattered. `arr[i]` computes `base + i * 8` and reads - once. No object dereference. No allocation per element.
The trunk of this book uses two containers: `list` for the small bookkeeping (the names of your tables, the schedule of your systems) and `numpy.ndarray` for the rows. There are no `dict`s of objects, no class hierarchies, no `dataclasses` with `__slots__` for the things that need to scale. Not because they don't exist, but because every container that wraps a `PyObject` per row pays the pointer-chase tax on every read, and the rest of the book is about not paying that tax.
## The flip, measured
Take the same data - N rows, K integers per row - and lay it out five ways. The first two are what the official tutorial teaches. The middle two are stdlib-only flips. The fifth is the disciplined endpoint.
| layout | what it is |
|-------------------------------------------------|-----------------------------------------|
| 1. `[(i, i+1, …) for i in range(N)]` | list of tuples - AoS, default |
| 2. `[[i, i+1, …] for i in range(N)]` | list of lists - AoS, mutable inner |
| 3. `tuple([i+k for i in range(N)] for k …)` | tuple of lists - SoA, stdlib |
| 4. `tuple(array.array('q', …) for k …)` | tuple of `array.array` - SoA, stdlib typed |
| 5. `tuple(np.arange(...) for k in range(K))` | tuple of numpy columns - SoA, typed + C |
[`code/measurement/aos_vs_soa_footprint.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/aos_vs_soa_footprint.py) builds each, in a fresh subprocess so RSS readings don't bleed, with N=1,000,000 and K=10. Values past the small-int cache so `PyLong` objects aren't shared singletons across rows. Three numbers per layout: peak RSS, construction time, time to sum column 0.
| layout | RSS | build | sum c0 |
|-------------------------------------|--------:|-------:|-------:|
| list of tuples (AoS) | 437 MB | 0.74 s | 24.9 ms |
| list of lists (AoS) | 498 MB | 0.61 s | 26.9 ms |
| tuple of lists (SoA) | 383 MB | 0.46 s | 2.5 ms |
| tuple of `array.array` (SoA typed) | 77 MB | 0.66 s | 11.6 ms |
| tuple of numpy int64 cols (SoA numpy) | 94 MB | 0.09 s | 0.4 ms |
> [!NOTE]
> Measured on this author's machine; reproduce on yours with `uv run code/measurement/aos_vs_soa_footprint.py`. Order-of-magnitude is the durable claim. Numbers will shift with K, N, value range, and CPython version, but the shape - that the AoS-to-SoA flip and the boxed-to-typed flip and the Python-loop-to-C-loop flip are three independent wins - is stable across machines.
The five rows separate three independent decisions that the four-row version conflated.
**The mutable AoS is worse than the immutable AoS.** Replacing the inner tuples with lists costs ~60 MB of additional list-header overhead at this scale. The "list of lists" pattern is the most-taught layout in introductory Python and the most-expensive one in this comparison.
**Step one - AoS → SoA - is the speed flip.** Tuple-of-lists is the same code an intermediate Python programmer might write without ever touching numpy. It saves only ~12% on memory but sums column 0 about **10× faster** than the AoS forms. The win is the access pattern: walking *one* contiguous list of 1M `PyLong` pointers instead of walking 1M tuple objects and dereferencing through each one to reach `row[0]`. Storage is barely better; the loop is dramatically better.
**Step two - boxed list → typed bytes - is the memory flip.** Going from `list[int]` to `array.array('q', …)` shrinks each column from ~38 MB of pointers-and-`PyLong`-objects to ~8 MB of contiguous int64 bytes. The whole structure drops to **~77 MB total**, smaller than numpy in this run (numpy carries ~20 MB of one-off import overhead). But the column-sum *slows down* - 2.5 ms → 11.6 ms - because Python has to *unbox* each `int64` into a temporary `PyLong` before adding it. The unboxing tax buys back about a third of the SoA speed win. **Typed storage saves bytes; it does not save the inner loop.**
**Step three - Python loop → C loop - is the order-of-magnitude move.** `np.sum` walks the same typed bytes that `array.array` stored, but the loop is in C and the interpreter is stepped out of the way. 11.6 ms → 0.4 ms; about **30× speedup** on the same bytes, no further memory saving (and a small import-overhead cost). This is the layout the simulator (§11+) and every system after it depends on.
Read the three steps together: **the SoA flip is the speed move, the typed-storage flip is the memory move, the C-vectorisation flip is the speed move again at a larger scale.** Each is a separate decision; each can be taken without the others. Numpy happens to bundle the second and third into one library, which is why most teaching collapses them into "use numpy." The exhibit shows they are separate wins.
## The Python-default trap, named
The official tutorial is not wrong. It's optimised for *teaching the language*, not for teaching layout. The path it teaches looks like this:
1. Make a class for the row.
2. Put instances in a list.
3. Reach for `dataclass` when the class gets noisy.
4. Reach for `__slots__` when memory pressure shows up.
Each step is a local improvement and a global trap. Step 1 commits you to AoS. Step 2 puts pointers between the rows. Step 3 makes the AoS more ergonomic. Step 4 saves a per-instance `__dict__` but does nothing about the fundamental shape - every row is still its own heap object reached through a pointer. The `__slots__` win is real and small; the SoA win is the same data costing 4-5× less memory, and you don't need a class at all.
There is no such thing as a cost-free abstraction. Every pointer has a cost, and in a `list` of rows that cost multiplies linearly with the row count. The four-step path stacks pointers: an outer list of N row-pointers, each row pointing to K field objects, each field a separately allocated value somewhere else on the heap. `__slots__` removes one layer (the per-instance `__dict__`); the SoA flip removes the rest. The next several phases of this book teach the alternative.
## Exercises
1. **Pointer-chase or value-read.** Print `sys.getsizeof(0)`, `sys.getsizeof(1000)`, `sys.getsizeof(10**100)`. Note that even a small Python int costs 28 bytes. Now print `np.array([0, 1000, 10**18], dtype=np.int64).nbytes`. Three int64s = 24 bytes, and there are no per-element headers.
2. **The interning trap.** Repeat exercise 1 with values 0 and 1, then again with values 257 and 1000. Use `id()` to confirm that `[0] * 1_000_000` shares one `PyLong` object across all positions, but `[1000 + i for i in range(1_000_000)]` does not. The "list of small ints is cheap" intuition only holds inside CPython's small-int cache `[-5, 256]`.
3. **Capacity vs length.** Build `lst = []`. In a loop, append 0..1000 and print `len(lst)` and `sys.getsizeof(lst)` after each step. Observe the over-allocation pattern - `list` grows in chunks, like `Vec::push`, but the chunks are CPython implementation detail (currently `~1.125 ×` growth).
4. **Run the §3 exhibit.** `uv run code/measurement/aos_vs_soa_footprint.py`. Read the output. The sum-c0 column matters: even if you ignore the memory line, the column-sum cost gap between layouts 1 and 4 is two orders of magnitude on the same data.
5. **The dict trap.** Build `d = {i: i*i for i in range(1_000_000)}` and time looking up 100,000 random keys. Build `arr = np.arange(1_000_000) ** 2` and time the same access pattern via `arr[idx]`. Note that you have replaced "look up by integer" with "index by integer," and the structures cost different amounts.
6. **swap-remove vs remove.** Build `lst = list(range(1_000_000))`. Time removing 100 elements from the middle by `lst.pop(500_000)` (slow - every pop shifts ~half the list). Time the equivalent via `lst[i] = lst[-1]; lst.pop()`. Note the orders-of-magnitude difference. This trick will earn its keep at [§21](#21---swap_remove).
7. *(stretch)* **Read your own array.** Use `np.frombuffer(arr.tobytes(), dtype=np.int64)` and confirm that `arr.data.tobytes()` is exactly `arr.size * 8` bytes long. The bytes you would write to disk *are* the bytes already in memory. This is what [§36 - persistence](#36---persistence-is-table-serialization) means by "tables serialise themselves."
## Applied reference
If you want to see this discipline carried through a real piece of code, read [`.archive/simlog/logger.py`](https://github.com/root-11/intro-book-python/blob/main/.archive/simlog/logger.py). It is a 700-line columnar logger that parks dict payloads into pre-allocated numpy columns, with a double-buffered design that lets the simulation write to one buffer while a background thread dumps the other to disk. The book does not require you to read it now. It's the destination this chapter and the next several point at.
## What's next
[§4 - Cost is layout, and you have a budget](#4---cost-is-layout---and-you-have-a-budget) takes the layout reasoning into per-tick territory: how many bytes can you actually move per tick on your machine, and what does that buy you in entities? After that, [§5 - Identity is an integer](#5---identity-is-an-integer) is where the through-line simulator gets its first concrete shape.
# 4 - Cost is layout - and you have a budget
A program runs at some *target rate*. A game runs at 30 Hz or 60 Hz; an audio loop at 48 kHz; a control loop at 1 kHz; a web request handler at "as fast as a human is willing to wait". The target rate sets a *budget* - the time available for one tick of work.
| Target rate | Budget per tick |
|----------------:|----------------:|
| 30 Hz | 33 ms |
| 60 Hz | 17 ms |
| 1000 Hz | 1 ms |
| 1 000 000 | 1 µs |
Every operation the program does in one tick spends from that budget. Operations have very different costs. From the numbers you measured in [§1](#1---the-machine-model):
| operation | typical cost |
|----------------------------:|-------------:|
| float multiply | < 1 ns |
| L1 read | ~1 ns |
| L3 read | ~10 ns |
| **Python interpreter dispatch** | **~5 ns / element** |
| RAM read | ~100 ns |
| disk read | ~100 µs |
| network round-trip | ~100 ms |
The bolded row is the one most explanations leave out. Inside a Python `for` loop, every step pays for `PYTHON_NEXT_INSTR`, refcount work, `PyObject` boxing - about 5 ns even when you do nothing. That cost is *higher than an L1 read* and competitive with an L3 read. It is the dominant fact about pure-Python performance, and it does not appear in any C-style cost table.
## Three regimes - and a fourth
A loop is **compute-bound** when its cost is dominated by arithmetic - typically when the data fits in L1 and the inner work is heavy (dot products, transcendentals, integer divides). It is **bandwidth-bound** when its cost is dominated by how fast the memory subsystem can deliver bytes - typically when the working set is bigger than L3 *but* the access pattern is sequential, so the prefetcher can fill lines ahead of demand. It is **latency-bound** when its cost is dominated by individual memory round-trips - typically when the access pattern is random, so the prefetcher cannot help.
Python adds a fourth: **interpreter-bound**. From the §1 cache-cliffs exhibit, summing 100 million `int64` values cost 4.59 ns per element in a Python list and 0.15 ns per element in a numpy array. The Python list run was not bandwidth-bound, nor latency-bound - the bytes were the same bytes. It was *interpreter-bound*. The CPU spent most of its cycles inside the bytecode dispatcher and the `PyLong` arithmetic path, not on the data. The fix is not "buy faster RAM"; the fix is *leave pure Python for the inner loop*.
The four regimes have very different time budgets:
| regime | cost per element | budget at 30 Hz |
|----------------------:|-----------------------:|-----------------------:|
| compute-bound | ~1 ns (L1 + ALU) | 33 million ops / tick |
| bandwidth-bound | ~0.2 ns (numpy seq) | 165 million ops / tick |
| latency-bound | ~12 ns (numpy gather)| 2.7 million ops / tick |
| interpreter-bound | ~5 ns (Python loop) | 6.6 million ops / tick |
A loop processing 1,000,000 entities in a 30 Hz tick costs 0.6% of the budget if it is bandwidth-bound, 36% if it is latency-bound, and 14% if it is interpreter-bound. *The same algorithm, the same data, four ways of running it, four orders of magnitude apart.* Complexity-class reasoning cannot tell these regimes apart.
## Cost is layout, not just complexity
The same algorithm that costs 0.2 ms on a sequential numpy column may cost 27 ms on a list-of-tuples carrying the same data, because every row read is a pointer chase to a separately allocated tuple, and every column read inside the row is another pointer chase to a `PyLong`. From the §3 exhibit, summing column 0 of one million ten-int rows took 30 ms as a list of tuples and 0.4 ms as a numpy SoA - a **75× spread on the same payload**. Two programs with the same big-O, same input data, and the same machine differ by almost two orders of magnitude on the inner loop, just because of where their data sits.
This gives you a design rule. *Decide your target rate before you decide anything else.* That sets the budget. Then when you choose data structures, ask whether the resulting working set fits in cache; ask how many memory loads per row your inner loop does; ask whether any single operation in the loop dominates the budget; **ask whether you are running inside the interpreter or outside it.** Most decisions become forced once the budget is named.
The reverse direction is also useful. If you find yourself wanting to *add* something to the inner loop - a dictionary lookup, a `getattr` against a class, a Python-level callback, an exception handler - count its cost in microseconds against the budget. Often the answer is "this single addition uses 80% of my tick", and the right move is not to optimise it but to lift it out of the inner loop entirely.
## The engineering analogy
The shape of this thinking is familiar to engineers in other domains. An electrical engineer designs a circuit by counting milliamps against a current budget. A structural engineer counts kilonewtons against a load budget. The data-oriented programmer counts microseconds against a tick budget. *Good design is measured in millivolts and microamps* - and in nanoseconds and microseconds. Pick the unit, write the budget down, count against it. Programming has no special exemption from accounting.
> [!NOTE]
> *Time is one budget. Power is another.* Cache hits are energetically nearly free - the data is already next to the arithmetic units. Cache misses fire up the memory controller, the bus drivers, sometimes a DRAM refresh; that is where the watts go. A loop that fits in L2 spends most of its time on cheap arithmetic; a loop that pointer-chases through RAM spends most of its time *waiting*, and during the waiting the CPU drops clocks and the chip stays cool. The same SoA-and-sequential-access discipline that fits the time budget also fits a power budget. For embedded, mobile, control, and battery-powered work, power is the *primary* budget; time is downstream of it. The "millivolts and microamps" line above is literal, not metaphor.
>
> One Python-specific addendum: an interpreter-bound loop is also relatively *power-hungry* per useful operation, because the CPU is running flat-out doing dispatch work instead of arithmetic. Moving to numpy improves time *and* energy at the same time. There is no trade-off here - the disciplined choice is also the cheap one.
## Exercises
1. **Pick your rates.** For each of these systems, name a plausible target rate and the resulting per-tick budget: a card game; a real-time strategy game; a market data feed; an embedded sensor controller; a web API endpoint a user is waiting for; an offline batch job that processes a billion rows.
2. **Count an operation.** Time a single `dict[k]` lookup on a dict of 1,000,000 entries (use `timeit` for a million repeats and divide). Note its cost in microseconds. How many can you fit in a 30 Hz tick (33 ms)? In a 1 kHz tick (1 ms)?
3. **The layout difference.** Sum 1,000,000 `int64` values in a numpy array. Sum 1,000,000 ints in a Python `dict` with integer keys (use `sum(d.values())`). What is the per-element time difference (in nanoseconds)? Where did it go? Map the answer back to the regime table above.
4. **The cliff.** With your numbers from [§1 exercise 2](#1---the-machine-model#exercises), pick a numpy array size that just fits in L2 and one that just doesn't. Time a `arr.sum()` at each size. The cliff is real.
5. **Working backwards from the budget.** You target 60 Hz; your inner loop runs over 100,000 entities; each entity touches one cache line of state. Estimate the cost of the loop in microseconds in each of the four regimes (compute, bandwidth, latency, interpreter). Compare to your 60 Hz budget (16,666 µs). Note which regime gives you headroom and which blows the budget.
6. **A bad design.** Construct a Python design that is "obviously fast" by big-O reasoning but blows the 30 Hz budget on a million entities. (Hint: list of `dataclass` instances with a per-tick `for entity in entities: entity.update()` is the canonical example. Estimate its cost from the interpreter-bound row of the regime table.)
7. **Find your CPU's TDP.** Look up your CPU's rated thermal design power on the manufacturer's spec sheet, or read it locally on Linux with `sudo dmidecode -t processor | grep -i 'power\|TDP'`. Note the value. TDP is what the chip can dissipate sustained without thermal throttling - burst can be 1.5-2× higher for tens of seconds; sustained settles back to TDP.
8. **Battery budget.** A typical laptop battery holds about 50 Wh. Your simulator runs at 30 Hz and draws an average of 8 W (mostly memory bandwidth on the inner loop). How many hours of simulation does a full charge buy? If a layout change pushes more loads to RAM and raises the average draw to 14 W, how many hours then? Express the cost of the layout change as a percentage of battery life.
9. **Measure delta power.** In one terminal, run a sustained sequential numpy sum loop:
```python
import numpy as np
arr = np.arange(10_000_000, dtype=np.int64)
while True: _ = int(arr.sum())
```
In another terminal: `sudo perf stat -a -e power/energy-pkg/ -- sleep 30` reads the package-energy counter over 30 seconds. Run the same measurement with a *random gather* version (`arr[idx].sum()` with a shuffled `idx`) and an idle baseline. Convert each to average watts. The random-access run should draw more watts than the sequential one, which should draw more than idle. The gap between them is the energy cost of breaking the prefetcher.
10. *(stretch)* **Joules per access.** Approximate energies per memory read: L1 hit ≈ 0.1 nJ, L2 ≈ 1 nJ, RAM ≈ 30 nJ (rough; published numbers vary by chip and process). Estimate the total energy of summing 10 million `int64`s sequentially (mostly prefetched, near-L1 cost) versus by random indices (mostly RAM misses). Convert both to milliwatt-hours and express as a fraction of a 50 Wh battery. The absolute numbers are tiny; the *ratio* is what your battery life and your data-centre electricity bill care about.
## What's next
You now have the machine model (§1), the data widths (§2), the table primitive (§3), and the budget calculus (§4). The next section is the conceptual heart of the book: [§5 - Identity is an integer](#5---identity-is-an-integer). The card game is waiting.
# 5 - Identity is an integer
Hand a Python programmer fifty-two cards and tell them to write code that shuffles, sorts, and deals. Ask how long.
Most will start drawing classes. The "official" Python tutorial path leads here: define `class Card` with `__init__(self, suit, rank)`, then `class Deck` holding a `list[Card]`, then `class Hand`, then probably `class Player` and `class Game`. By the time the type hints are right and the `__repr__` methods print nicely, an evening has passed. There will be debates about whether `Hand` should *contain* `Card` instances or hold references to a shared `Deck`, whether `Deck.shuffle()` should mutate or return a new deck, whether `Card` should be a `@dataclass(frozen=True)` for hashability. None of these debates are wrong; all of them are work that has nothing to do with cards.
The whole problem fits in three lines of numpy. The way it fits is the lesson of this section.
A deck of cards has three pieces of information per card: its suit (♠ ♥ ♦ ♣), its rank (A, 2, ..., K), and its current location (in the deck, in someone's hand, in the discard pile). That is three columns. The deck itself is fifty-two rows.
```python
import numpy as np
suits = np.zeros(52, dtype=np.uint8) # 0..3
ranks = np.zeros(52, dtype=np.uint8) # 0..12
locations = np.zeros(52, dtype=np.uint8) # 0=deck, 1..N=hands, 255=discard
```
That is the deck. The whole thing is **156 bytes** - three contiguous columns of 52 unsigned bytes. There is no `Card` class. There is no `Deck` class. The card at index `17` has its suit at `suits[17]`, its rank at `ranks[17]`, and its current location at `locations[17]`. The card *is* the index.
Filling the columns with a fresh, ordered deck is one assignment per column:
```python
suits[:] = np.repeat(np.arange(4, dtype=np.uint8), 13)
ranks[:] = np.tile(np.arange(13, dtype=np.uint8), 4)
locations[:] = 0
```
Dealing card 17 to player 1 is one element write:
```python
locations[17] = 1
```
Asking *what's in player 1's hand* is one numpy primitive:
```python
hand = np.where(locations == 1)[0]
```
`hand` is a numpy array of indices into the deck - a *list of card identities* - not a copy of any card data. Asking *how many cards are in each location* is also one primitive:
```python
counts = np.bincount(locations, minlength=2) # counts[0] = deck, counts[1] = player 1, ...
```
Shuffling - the move students expect to be hard - is shuffling the order of indices. `0..52` becomes `[7, 32, 1, 19, ...]`, and you read your way through the cards in that order:
```python
order = np.random.permutation(52)
```
Look at what just happened. Nothing about the cards changed. `suits[17]`, `ranks[17]`, and `locations[17]` are exactly the values they were before. The shuffle moved indices, not data.
Sorting works the same way. To sort by suit then rank, you sort the indices by `(suits[i], ranks[i])`:
```python
order = np.lexsort((ranks, suits)) # last key is primary; sort by suit first, then rank
```
The cards do not move. Their identifiers are reordered.
That's the deck of cards in maybe fifteen lines of Python. It includes shuffle, sort, deal, and several queries. It is not a stylistic shortcut; it is what a deck of cards *is*. The class-hierarchy version's evening of work was the cost of pretending a card was an object that owned its suit and rank, when actually a card is one number - an index - and its suit and rank are values stored in arrays at that index.
We call this **identity-is-an-integer**, and it is the precondition for every economy the rest of this book buys you. Persistence will work because tables are easy to serialise - three `np.save` calls. Parallelism will work because indices are cheap to partition. Replay will work because a deck is just three arrays in a state. None of it works if you reach for `class Card`.
## Even *which* integer matters
Not every integer is the same integer for performance. From [`code/measurement/float_or_int_tuple.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/float_or_int_tuple.py), looking up keys in a Python `dict` of 10,000 entries:
| key shape | lookups / sec |
|----------------------------|--------------:|
| `(int, int)` | 42,800,637 |
| `(int, int, int)` | 39,625,273 |
| `(float, float)` | 26,461,898 |
| `(float, float, int)` | 26,115,850 |
| `(float, float, float)` | 17,630,435 |
A two-tuple of ints hashes and compares **2.4× faster** than a three-tuple of floats. Identity-is-an-integer is not just "use a number"; it is "use a small unsigned integer, ideally in a contiguous typed array." A `np.uint8` index packs 64 to a cache line and hashes in one CPU instruction. A `(float, float, float)` "identity" - the kind a Python tutorial might suggest for a 3D point in a dict - pays the price three times: more bytes, slower hash, slower compare.
The card-deck columns above use `np.uint8` deliberately: 0..255 covers everything (4 suits, 13 ranks, up to 254 locations), one byte per value, 64 cards per cache line. The width budget from §2 meets the identity choice from §5: a `np.uint8` column is the cheapest possible identity, the cheapest possible storage, and the cheapest possible lookup, all in one decision.
> [!NOTE]
> *The strong form, which we will return to later:* sometimes you do not even need the index. The pair `(suit, rank)` already uniquely identifies a playing card - there are only fifty-two such pairs. The index is a *surrogate key*; the pair is a *natural key*. For variable-quantity tables (creatures that come and go) you usually need a surrogate, because two creatures can be identical. For a constant-quantity 52-card deck, you do not. The book uses surrogates throughout because the simulator is variable-quantity, but knowing when you can drop the index is its own discipline.
## Exercises
The first time through, write everything from scratch in `deck.py`. Resist the urge to add a `Card` class or helper methods. Three numpy arrays.
1. **Build the deck.** Write `def new_deck() -> tuple[np.ndarray, np.ndarray, np.ndarray]` that returns the suits, ranks, and locations for a fresh, ordered deck (all 52 in `location 0 = deck`). All three arrays are `dtype=np.uint8`.
2. **Print a card.** Write `def card_to_string(suit: int, rank: int) -> str` that returns strings like `"A♠"`, `"10♥"`, `"K♦"`. Use it to print the whole deck.
3. **Shuffle.** Use `np.random.default_rng(seed).permutation(52)` to produce a shuffled order. Print the deck in shuffled order. Confirm by inspection that the `suits`, `ranks`, and `locations` arrays are unchanged.
4. **Sort by suit then rank.** Use `np.lexsort((ranks, suits))` to produce an `order` such that suits come out grouped, ranks ascending within each suit. Print again. Once again, the deck arrays are unchanged.
5. **Deal a hand.** Move the first 5 cards from the deck (location 0) to player 1 (location 1). Print player 1's hand using `card_to_string`.
6. **Hand query.** Write `def cards_held_by(locations: np.ndarray, player: int) -> np.ndarray` returning all card indices currently held by a given player. The body is one line.
7. **Count by location.** Write a function that returns counts grouped by location using `np.bincount`. Confirm `counts[0] + counts[1:].sum() == 52`.
8. **Deal four hands.** Deal 5 cards to each of players 1, 2, 3, 4. Print all four hands.
9. *(stretch)* **Drop the index.** Rewrite `cards_held_by` to return an `(N, 2)` numpy array of `(suit, rank)` pairs directly - no indices. What does this make easier? What does it make harder? (Hint: you cannot move the cards back to the deck without knowing which `i` they were.)
10. *(stretch)* **The sort hazard.** While player 1 is holding indices `[3, 17, 21, 28, 41]`, sort the deck arrays *themselves* in place by suit (`order = np.argsort(suits); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]`). What does player 1 think they hold now? Print the cards at the indices `[3, 17, 21, 28, 41]` after the sort. This is the bug [§9 - sort breaks indices](#9---sort-breaks-indices) was written for. Don't fix it yet - observe it.
Reference solutions for exercises 1-3 in [05_identity_is_an_integer_solutions.md](https://root-11.codeberg.page/intro-book-python/trunk/05_identity_is_an_integer_solutions.html). Solutions for the rest follow the same shape.
## What's next
Exercise 10 leaves you with a bug. The next several sections build the discipline that prevents it: [§6 - A row is a tuple](#6---a-row-is-a-tuple) is the next vocabulary lesson, and [§9 - sort breaks indices](#9---sort-breaks-indices) is the fix - keep a stable id alongside the position so external references survive reordering.
# 6 - A row is a tuple
In §5 you built a deck of 52 cards as three numpy columns. The card at index 17 is the triple `(suits[17], ranks[17], locations[17])`. Together those three values are *the row*. There is no `Card` class. There is not even a tuple object - the row exists *implicitly* in the alignment: the same index, used in every column, recovers all the data about one card.
This is what we call a *row* throughout the rest of the book - a coherent set of values that belong to the same entity. In a `creature` table the row is `(pos[i], vel[i], energy[i], birth_t[i], id[i], gen[i])`. In a `food` table it is `(pos[i], value[i], id[i])`. The fields belong to the same entity by virtue of all sharing index `i`. There is no `dataclass` holding them; there is no `NamedTuple` instance; there is no `dict`. There is only the discipline that whatever index `i` you used to read one column, you also use to read every other column of the same table.
## Why "implicit" matters in Python
Python's tutorial reflex when it sees the word *row* is to reach for a class - `@dataclass class Row` or `class Row(NamedTuple)` or, if performance is mentioned, `class Row: __slots__ = (...)`. Each of these constructs the row as an *object*, with a header, a refcount, and field pointers. None of them are free. From [`code/measurement/classes_or_tuples.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/classes_or_tuples.py), the time to materialise 1,000,000 two-field "rows" on this machine, ordered fastest to slowest:
| how the row is built | time for 1M rows |
|----------------------------------------------------------|-----------------:|
| numpy SoA - two `np.full(N, value)` columns (bulk) | 0.005 s |
| `(x, y)` - bare tuple, 1M individual constructions | 0.007 s |
| `class` with `__slots__` | 0.109 s |
| `collections.namedtuple(...)` | 0.146 s |
| `typing.NamedTuple` subclass | 0.151 s |
| `@dataclass(frozen=True, slots=True)` | 0.164 s |
Two readings of this table.
First reading: the bare tuple is **~16× faster** than a slotted class and **~23× faster** than a frozen+slots dataclass for per-row construction. The named alternatives all pay for an object header and per-field descriptor lookup that the tuple skips. From [`code/measurement/simple_namespace.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/simple_namespace.py), even a `dict` (`{'x': 10.0, 'y': 20.0}`) constructs faster than any of the named-class options - about 0.036 s for the same million. *Naming the row* is the cost; the tuple is the cheapest row that is still recognisable as a row.
Second reading - and the one this book cares about - is the top line: **two bulk numpy column allocations construct 1,000,000 rows-worth of data faster than a million individual tuple literals.** Bulk allocation is roughly 30× faster than the named alternatives and is *not even slower than the cheapest per-row option*. The shape that lets you do this - pre-allocate a column once, fill it with values, and treat row `i` as the implicit tuple `(col0[i], col1[i], ...)` - has no per-row construction cost at all. The tuple at index `i` only exists when you ask for it explicitly; until then it lives in contiguous bytes inside numpy columns. From the §3 footprint exhibit, one million ten-field rows cost 99 MB as numpy SoA columns and 437 MB as a list of tuples - and the SoA version pays *zero* per-row construction cost on top of that, because there are no row objects.
A row is a tuple, but in Python the most useful version of that statement is: **a row is a tuple you do not have to build.**
## Alignment is the discipline
The cost of implicit binding is that you must *keep the indices aligned*. If you sort `suits` without also sorting `ranks` and `locations`, the row at every index is corrupted - the deck still has 52 entries in 52 slots, but each slot now holds the suit of one card, the rank of another, the location of a third. This is not a hypothetical bug; you produced it deliberately in §5 exercise 10, and [§9](#9---sort-breaks-indices) will hand you the structural fix. The rule is simple: *every operation that reorders any column of a table must reorder all columns of that table together.*
The discipline that makes alignment maintainable is **single-writer-per-column**. If only one function writes to `locations`, and that function writes consistently, alignment is never violated. Multiple writers to the same column race against each other and produce inconsistent rows. This is what [§25 (one writer, many readers)](#25---one-writer-many-readers) enforces: each table has exactly one writer, and a row is a tuple precisely because that one writer kept all its columns in step.
A row is a tuple - assembled from columns indexed by the same entity, kept aligned by discipline rather than by any container holding it together.
## Exercises
These extend your `deck.py` from §5.
1. **Print row 17.** Write `def row(suits, ranks, locations, i)` returning `(int(suits[i]), int(ranks[i]), int(locations[i]))`. Use it to print the suit, rank, and location of card 17.
2. **Mishandle the alignment.** Sort *only* `suits` in place: `suits.sort()`. Print row 17 again. The values are now from three different cards - exactly the bug.
3. **Lockstep sort.** Reset the deck. Now sort all three columns *together* using an order array: `order = np.argsort(suits); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]`. Print row 17 again. The values are from one card. (The `[:]` matters - it is an in-place assignment that keeps the same backing array; `suits = suits[order]` would rebind the name to a new array and break aliases held elsewhere.)
4. **Add a fourth column.** Add `dealt_at = np.full(52, 255, dtype=np.uint8)` (when a card is dealt at tick `t`, write `t` into `dealt_at[i]`; the sentinel 255 means "not yet dealt"). Modify your lockstep sort to also reorder this column. Verify by spot-check that a row is still consistent after a sort.
5. **The single-writer rule.** Write `def reorder_deck(suits, ranks, locations, dealt_at, order)`. This function is the *only* one that should ever reorder any column of the deck. Document that contract in a docstring above the function. Refactor your shuffle and sort to call it.
6. **The construction cost, your machine.** Run `uv run code/measurement/classes_or_tuples.py` on your machine. Note the ratios. Confirm that the slotted-dataclass row, the canonical "right" answer in modern Python, is the *slowest* of the named options at construction.
7. *(stretch)* **When alignment is moot.** A query that uses only `(suits[i], ranks[i])` to identify a card - for instance, "is this the Ace of Spades?" - does not depend on `locations` or `dealt_at`. Write such a query (one line, using `np.where`). The natural-key view from §5's strong form means this query survives reorderings of unrelated columns; only `suits` and `ranks` need to be aligned with each other.
## What's next
[§7 - Structure of arrays (SoA)](#7---structure-of-arrays-soa) names the layout choice you have been making implicitly: each field its own column. The next section defends that choice against its alternative.
# 7 - Structure of arrays (SoA)
Your deck has three numpy columns: `suits`, `ranks`, `locations`. Each field lives in its own array, indexed by entity. This layout is called *Structure of Arrays* - SoA. The opposite layout - a single `list[Card]` where each element is a `dataclass` holding all three fields - is called *Array of Structs* - AoS. They are different choices about *where the same data lives*.
```python
# SoA: three columns, indexed in lockstep
suits = np.zeros(52, dtype=np.uint8)
ranks = np.zeros(52, dtype=np.uint8)
locations = np.zeros(52, dtype=np.uint8)
# AoS: one list of objects
@dataclass
class Card:
suit: int
rank: int
location: int
cards: list[Card] = [...] # 52 instances
```
Most Python programmers reach for AoS by default. It is what every introductory tutorial teaches: define a class for the entity, put instances in a list. The trouble is that in a real loop "the entity" is whatever the inner loop reads, not whatever the data model says belongs together. A system that counts cards in player 1's hand reads only the location column - it does not need suits or ranks at all.
## What "reads only one column" actually costs
With SoA, that count is one numpy primitive:
```python
held_by_p1 = int(np.sum(locations == 1))
```
That call walks **N bytes** of `locations`, generates an N-byte boolean mask, and sums it - all inside C, no Python-level iteration. At N = 1,000,000 cards on this machine, the call takes ~0.5 ms.
With AoS, the same count is a Python `for` loop:
```python
held_by_p1 = sum(1 for c in cards if c.location == 1)
```
That loop pays for one bytecode dispatch per card, one `getattr` per card, one comparison per card, and one increment per card. From §1, interpreter dispatch is ~5 ns/element, and `getattr` adds more. At N = 1,000,000 the same count takes 30-50 ms - **two orders of magnitude slower** for the identical answer on the identical data.
This is the bandwidth-bound vs interpreter-bound regime distinction from §4. SoA pushes the inner loop into C and walks contiguous bytes; AoS keeps the inner loop in the interpreter. The SoA call can run inside a 30 Hz tick (33 ms budget) at 1 million entities and use under 2% of the budget. The AoS call uses the entire tick budget at 1 million entities, leaving no room for the rest of the simulation.
## The Python AoS penalty does not shrink with width
In a Rust AoS layout, the cost grows with the size of the struct: a 19-byte `Card` fills a cache line with three cards instead of sixty-four bytes of locations. A reader who does not need suits and ranks pays for them anyway because they ride in on the same cache line. Add a 16-byte `nickname` field and the gap widens.
In Python the story is different. Every field of a `dataclass` is a `PyObject*` pointer, so a "wider" `Card` does not put more *bytes* in the same cache line - it puts more pointers. The cost of `c.location` is not "extra cache traffic"; it is the fixed overhead of the Python attribute lookup. Adding fields you do not read makes each `Card` heavier in absolute terms (more allocation, more refcounts) but does not slow down the per-attribute access. The penalty is *fixed* by interpreter dispatch and `getattr`.
This makes the SoA win in Python *categorical*, not just *quantitative*. The numpy primitive escapes the interpreter entirely; the AoS loop does not. No amount of `@dataclass(slots=True)` discipline removes the per-attribute dispatch cost. From §6, slots reduce *construction* cost and per-instance memory, but every read of `c.location` still goes through Python's attribute machinery.
## SoA is the default
SoA is therefore the default in this book. AoS is sometimes the right choice - for example when every system reads every field of every entity on every tick (rare), or when N is so small that the loop overhead dominates regardless of layout (think dozens of items, not millions). But this is a tradeoff to *earn* by measurement, not to assume by habit. Write SoA first; switch to AoS only when a benchmark forces you to.
The §3 exhibit ([`code/measurement/aos_vs_soa_footprint.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/aos_vs_soa_footprint.py)) is the reference measurement for this chapter. Re-read its sum-column-0 row: list-of-tuples (the AoS twin) summed column 0 of one million ten-field rows in 30 ms; numpy SoA did the same in 0.4 ms. **75× faster for the canonical "system reads one column" operation.** That is the regime your inner loops will live in for the rest of this book.
> [!NOTE]
> **numpy stores rows; pandas stores columns.** numpy arrays are row-major (C order) by default, with column-major available via `order="F"`. pandas is the other way - column-oriented under the hood (each column stored contiguously), which is why a DataFrame is fast down a column and slow across a row. Neither layout is optimal for all uses: SoA wins when loops read a few fields across many rows; row storage wins when loops touch whole records one at a time.
## Exercises
You will need `time.perf_counter()` for some of these.
1. **Build both layouts.** Take your `deck.py` from §5 and add an AoS twin: a `list[Card]` of 52 entries, where `Card` is a `@dataclass` with three int fields. Build both and verify they encode the same logical content.
2. **Count cards in a player's hand, both ways.** Write `count_held_soa(locations, player)` using `np.sum(locations == player)` and `count_held_aos(cards, player)` using a Python generator expression. Confirm they return the same number on the same deck.
3. **Time the count at 10,000 entries.** Replicate your deck to length 10,000. Time both functions with `timeit` (e.g., `number=1000` for the numpy one, `number=100` for the AoS one). Note the ratio in nanoseconds per element.
4. **Scale to 1,000,000 entries.** Repeat at length 1,000,000. The SoA version reads 1 MB of bytes; the AoS version walks a million pointer-chases through Python's attribute machinery. Note the ratio. On most machines it is in the 50-200× range.
5. **The unused-field case, Python edition.** Extend `Card` with a `nickname: str = ""` field and a `dealt_at: int = -1` field - five fields total instead of three. Rebuild both. Time the count again. Note that the **SoA time is unchanged** (the count still walks only `locations`) and the **AoS time is also roughly unchanged** (interpreter dispatch dominates either way). Compare to the Rust version of this chapter, where the AoS time *grows* with row size - Python's penalty is fixed differently.
6. **A case where AoS does not lose.** Write a function that updates *every* field of one specific card. SoA writes to three (or five) different columns; AoS writes to one Python object. For the case "update every field of one card" - single entity, no loop - AoS is competitive or better. Time it. Note that this case has no inner loop, which is why the regime distinction from §4 doesn't apply.
7. **Construct, then read.** From §6 you know constructing `dataclass` instances is slow. Time *building* a million-entry AoS list once, then summing the location query 1000 times. Compare to building a million-entry SoA once, then summing 1000 times. The construction cost amortises over many reads; for short-lived data, even SoA construction time becomes a factor. (Hint: this is a foreshadowing of [§22 - mutations buffer](#22---mutations-buffer-cleanup-is-batched).)
8. *(stretch)* **A from-scratch `SoaDeck` class.** Wrap the columns (suits, ranks, locations, dealt_at) in one Python class that owns them all. Provide `reorder(self, order)` as the only public mutator. What do you gain in correctness? What do you lose in flexibility? (Hint: you have just rebuilt the contract from [§25 - one writer, many readers](#25---one-writer-many-readers), four chapters ahead of schedule.)
## What's next
[§8 - Where there's one, there's many](#8---where-theres-one-theres-many) is the universalising principle. The deck taught it implicitly; the next section names it.
# 8 - Where there's one, there's many
Code is written for the array. A function that operates on one entity is just the special case of N = 1; it does not need its own abstraction. A card game with 52 cards is three arrays - suit, rank, location - not 52 objects. A simulation with 100 creatures is six arrays of length 100, not 100 instances of `Creature`. The plural is the primary unit; the singular is the trivial case.
The pattern is simple. Write the array version first. The singleton drops out as a one-element slice. To shuffle one card you swap two indices in the `order` array - same as shuffling the whole deck. To find the highest-rank card in player 1's hand you scan the (small) hand array - same shape as scanning all 52. To deal one card you write one cell in `locations` - same shape as dealing many cells.
## The OOP instinct, named
This stands against an instinct most Python programmers acquire on day one: the urge to write `card.shuffle()` or `creature.update()` and then puzzle over how to do it for many. Almost every Python tutorial models behaviour as methods on objects, then introduces lists of objects as the natural way to *have many*, then introduces `for c in creatures: c.update()` as the natural way to *do something for each*. Three steps, each locally sensible, that together build the pattern this chapter is asking you to drop.
The puzzle does not exist when you write for arrays from the start. `shuffle(deck)` is one function that works for any deck, including a deck of one. `update(creatures)` - taking the columns as numpy arrays - is one function that works for any population, including a population of one. The method-on-object form is *strictly more code* than the function-over-slice form: it requires a class, an `__init__`, a `self` argument that does nothing useful at the array level, and a calling convention that prevents the inner loop from ever leaving the interpreter.
A useful test: when you find yourself writing a method on a class, ask *what does this look like over an array?* If the array version is shorter, drop the method. If the array version is the same length, keep it as a free function over numpy arrays - `def shuffle(suits, ranks, locations, order)`, not `class Deck: def shuffle(self): ...`. Either way, the singleton was never the right unit of code.
## The performance argument
There is also a performance reason - sharper in Python than in any compiled language. A method that operates on one entity at a time forces the system that uses it to call the method N times. From [`code/measurement/cache_cliffs.py`](https://github.com/root-11/intro-book-python/blob/main/code/measurement/cache_cliffs.py), Python per-element work cost ~5 ns regardless of the size of the data; numpy bulk work cost ~0.2 ns/element. The ratio is **roughly 25×** at any size, and that is *just* the dispatch cost - before you add the cost of `getattr(creature, 'energy')` once per call, the refcount work on every return, and the lost opportunity for numpy to use SIMD instructions on contiguous bytes.
In a compiled language, an "obvious" inner loop over `creatures.iter().for_each(|c| c.update())` is something the optimizer can usually rescue - inline the method, fuse the body into the loop, autovectorize the result. In Python the optimizer is the bytecode dispatcher and it cannot do any of that. The per-method-call form is essentially the worst case the language offers. Writing for arrays first is a request the *interpreter* can fulfil - it can hand the work to numpy and step out of the loop entirely. Writing for singletons-and-iterate is a request that pins the work inside the interpreter for every element.
"Where there's one, there's many" is therefore not an architectural slogan but a daily practice. It costs nothing the first time. It costs everything the first time you forget.
## Exercises
These extend `deck.py` once more. The aim is to feel the array-first pattern in your fingertips before Part 3 turns into the rest of the book.
1. **The function over a slice.** Write `def highest_rank_in_hand(hand, ranks)` where `hand` is a numpy array of card indices and `ranks` is the deck's rank column. Body should be one line: `int(ranks[hand].max())`. Use it on a 5-card hand. Then use it on a 1-card hand. Then use it on an empty hand. Same function, three N values.
2. **Reverse the urge.** Given an OOP-style `def is_face_card(self) -> bool` that lives on a hypothetical `Card` class, rewrite it as `def face_cards(ranks)` returning a numpy boolean mask of shape `(N,)`. Apply it to all 52 cards in one call: `mask = face_cards(ranks); face_count = int(mask.sum())`.
3. **The N = 0 case.** What does `highest_rank_in_hand` do when `hand` is empty? `arr.max()` on an empty array raises. Pick a behaviour - return `None`, return a sentinel, raise - and justify the choice. (Hint: most uses can short-circuit with `if hand.size == 0: return None`.)
4. **Predicate over a single value.** Suppose you want `is_red(suit)` for a single card (suits 0 and 1 are hearts/diamonds). Write the array version `def red_mask(suits)` first - one line: `(suits < 2)`. Then convince yourself the singleton case is `red_mask(np.array([suit]))[0]` - the array version covers it.
5. **Count overhead.** Time `sum(is_face_card_per_row(suits[i], ranks[i]) for i in range(52))` against `int(face_cards(ranks).sum())`. The array version should be measurably faster at 52, much faster at 100,000. Document the ratio. (Repeat at N = 100,000 by replicating the deck.)
6. **The dataclass twin, revisited.** Take your `list[Card]` from §7 exercise 1. Write `face_count_aos(cards)` as a generator-expression sum and `face_count_soa(ranks)` as the numpy version. Time both at 1,000,000 entities. The ratio you measure here is the same ratio §7 measured for `count_held` - it is not specific to one query, it is the per-element dispatch cost of *any* inner loop you write in pure Python.
7. *(stretch)* **From a tutorial.** Find any Python tutorial that uses a `class Card` with methods (`__init__`, `is_face`, `__repr__`, etc.). Rewrite their full card game as three (or four) numpy arrays plus free functions. Compare line counts. Compare clarity. Compare what happens when you want to query "all face cards across the table" - one numpy call versus a loop over per-card method calls.
## What's next
You have closed Identity & structure. Cards behave; rows align; layouts are SoA; the singleton drops out. The next phase is *Time & passes*, starting with [§11 - The tick](#11---the-tick). The ecosystem simulator from `code/sim/SPEC.md` is about to start running.
# 9 - Sort breaks indices
In [§5 - Identity is an integer](#5---identity-is-an-integer), exercise 10 left you with a bug. Player 1 was holding the index list `[3, 17, 21, 28, 41]`. The dealer sorted the deck columns by suit. Player 1's hand was now wrong - the same indices, the same slots, but different cards.
That bug is the structural fact this section names. Sorting did not damage anything; the player's reference was never robust to begin with. **An index points at a *slot*, not at a *thing*.** When the slot's contents change, the index quietly changes meaning.
It is not only sorting. Any rearrangement does it: `swap_remove` (a O(1) deletion that moves the last row into the freed slot, coming in [§21](#21---swap_remove)), reshuffling for locality ([§28](#28---proximity-is-a-property-of-position)), compacting after a batch of deletions. The same index, the same array, the same line of code, now means a different card.
## "But Python objects are stable references - can't I just go back to that?"
This is the moment many readers feel the urge to retreat. The Python reflex from §6 - `class Card` with attributes - gave you object identity for free. A `Card` instance you held a reference to last week is still the same `Card` object today, regardless of what happened to the list it was in. `id(card)` does not change. The pointer through the Python interpreter to the heap-allocated `Card` is stable for the lifetime of the object.
So the temptation is real: keep the index-aligned numpy columns *and* a parallel `list[Card]` of object references, and use the objects when you need stability. Or just go back to `list[Card]` entirely - at least the references work.
This trade does not survive contact with the §3 footprint table or the §7 access-cost table. The numpy-SoA layout is **5× smaller** and **75× faster** at single-column queries than `list[Card]`; carrying a parallel object list to "rescue" reference stability gives back most of the footprint win and adds the synchronisation problem of keeping the column data in step with the object data. You have not solved the problem; you have hidden it inside an additional invariant.
The structural fix is the one [§10](#10---stable-ids-and-generations) builds: an `id` column that travels with the row across rearrangements, plus (for variable-quantity tables) a generation counter on top. **The card itself is a slot; the card's *name* is an integer that we choose to be stable.** The cost is one extra `np.uint32` column. The benefit is that every rearrangement we will need from now on - sort, swap_remove, locality-driven reordering, compaction - works without breaking outside references.
This section's only job is to make the *slot vs name* distinction concrete enough that §10's solution feels inevitable rather than ceremonial.
> [!NOTE]
> *Why feel the pain first?* Because the fix in §10 is small - one extra column - and small fixes only stick if the student knows what they fix. Reading "always store an id" without first feeling the bug produces students who add ids cargo-culted, then drop them when the codebase looks too cluttered. Reading it after watching player 1 lose their hand produces students who never drop them.
## Exercises
You should still have your `deck.py` from §5. These exercises extend it.
1. **Reproduce the bug.** With player 1 holding `[3, 17, 21, 28, 41]`, sort the deck columns themselves (`suits`, `ranks`, and `locations` in lockstep) by suit. The pattern is `order = np.argsort(suits, kind="stable"); suits[:] = suits[order]; ranks[:] = ranks[order]; locations[:] = locations[order]`. Print player 1's hand using `card_to_string`. Confirm the cards have changed.
2. **A second rearrangement.** Instead of sorting, swap two cards' positions:
```python
suits[[3, 17]] = suits[[17, 3]]
ranks[[3, 17]] = ranks[[17, 3]]
locations[[3, 17]] = locations[[17, 3]]
```
Print player 1's hand again. Same bug shape, different cause.
3. **A third rearrangement.** Remove the card at slot 7 with the `swap_remove` pattern (move the last row into slot 7, then drop the last row): `suits[7] = suits[-1]; suits = suits[:-1]` and likewise for the other columns. Print player 1's hand. Note that the cards at slots `[17, 21, 28, 41]` are unchanged but slot 3 may now hold what was previously the last card; meanwhile slot 51 has silently been deleted.
4. **Quantify the breakage.** Write a function that takes the original `[3, 17, 21, 28, 41]` plus a freshly built deck, applies a Fisher-Yates shuffle to the deck columns themselves (`order = rng.permutation(52)` and reorder all three columns), and counts how many of the five references still point at the same `(suit, rank)` value. Run it 100 times. Roughly what fraction of references survive a random shuffle of the deck? (Spoiler: very small. With probability `1/52` per slot, the expected number that survive by accident is `5/52 ≈ 0.1`.)
5. **A reference that *can* survive.** Without writing any new code - on paper - describe what kind of reference would survive a shuffle. (Hint: you already know. The card's `(suit, rank)` is unique to that card. The reference that survives is the one that does not depend on the slot.)
6. **The "object reference" non-fix.** Build a parallel `list[Card]` (use a `@dataclass` if you wish) alongside the numpy columns. Fill them so that `cards[i]` mirrors `(suits[i], ranks[i], locations[i])`. Now sort the numpy columns by suit *without* updating the object list. What does player 1 see if they read from the object list? What if they read from the numpy columns? Note that you have introduced a new bug - *desynchronised* state - without fixing the old one.
7. *(stretch)* **The cost of never rearranging.** Suppose you decide to *never* sort, swap, or remove from the deck columns, to avoid this bug forever. How would shuffling work? How would discarding a card work? Why does this not scale to ten thousand creatures?
Reference notes for these exercises in [09_sort_breaks_indices_solutions.md](https://root-11.codeberg.page/intro-book-python/trunk/09_sort_breaks_indices_solutions.html).
## What's next
Exercise 5 points at the answer; exercise 7 makes the never-rearrange option look bad. The real fix is to store identity *separately from position* - an `id` column that travels with the row across rearrangements, with a generation counter on top for variable-quantity tables. [§10 - Stable IDs and generations](#10---stable-ids-and-generations) builds it.
# 10 - Stable IDs and generations
In [§9](#9---sort-breaks-indices) you watched a player's reference go stale because they were holding *slots*, not *names*. The fix is to give each row a name - a stable identifier - that travels with the row when it moves.
A stable id is one extra column. For the deck:
```python
ids = np.arange(52, dtype=np.uint32)
```
Now every card has both a *slot* (its current index in the columns) and an *id* (its name). When you sort the columns, you reorder `ids` in lockstep with everything else:
```python
order = np.argsort(suits, kind="stable")
suits[:] = suits[order]
ranks[:] = ranks[order]
locations[:] = locations[order]
ids[:] = ids[order]
```
The card with `id == 17` is still the same card - its suit, rank, and location are unchanged. It is just at a different *slot*.
To find a card by id, scan the `ids` column:
```python
def slot_of(ids: np.ndarray, target: int) -> int | None:
matches = np.where(ids == target)[0]
return int(matches[0]) if matches.size else None
```
That is O(N), which is fine for a 52-card deck and slow for a million creatures. The fix - an `id_to_slot` map maintained on every rearrangement - is [§23 - Index maps](#23---index-maps). For now the linear scan is honest pedagogy.
## Generations: when slots are reused
The deck is constant-quantity. Always 52 cards, never more, never less. The simple `ids` column is enough.
For variable-quantity tables - creatures that are born and die, packets that arrive and are processed, sessions that come and go - slots get *reused*. A new creature is born in the slot that just held a dead one. The `ids` column for such a table behaves like an *auto-incrementing primary key* in a database: every new row gets a fresh, never-reused integer; old rows keep their original ids forever. The simulator differs from a database in one structural way - it recycles *slots* to keep memory bounded, while a database table just grows. That recycling is what generations exist for. Imagine code that held a reference to the dead creature: their reference points at a slot that may now hold a different creature with possibly the same id (if id reuse happens) or - worse - a *valid-looking* row that is no longer the row they cared about.
One more column fixes it: a `gens` (generation) counter that increments every time a slot is recycled. A reference is now a pair `(id, gen)`. To dereference it, you check that the row's stored `gen` still matches the reference's `gen`. If it does, the reference is live. If it does not, the slot has been recycled since the reference was taken, and the dereference returns `None`.
```python
from typing import NamedTuple
class CreatureRef(NamedTuple):
id: int
gen: int
def get_slot(creatures, ref: CreatureRef) -> int | None:
slot = creatures.id_to_slot.get(ref.id)
if slot is None:
return None
if int(creatures.gens[slot]) != ref.gen:
return None
return slot
```
(This is one of the few places in the book where a `NamedTuple` earns its weight: a `CreatureRef` is a value passed through external code, and giving it field names makes the API readable. Per §6, the cost is real - a `NamedTuple` allocation per reference - but references are rare, not per-tick. Where the same lesson runs through hot data, the answer is still numpy columns.)
This is the pattern called a *generational arena*. It is the single mechanism behind every "handle" type in every ECS engine: Bevy's `Entity`, Rust's `slotmap::SlotMap`, C++'s `entt::registry`, and the indirect-handle pattern in databases. They differ in details - width of the id, packing into a `u64`, generation overflow handling - but the structural idea is the same: one column for identity, one for generation, a checked dereference.
That is enough machinery for the rest of the book to lean on. Sorting now works because the id column travels with the row. Deletion now works because the generation counter rejects stale references. Append-only and recycling tables ([§24](#24---append-only-and-recycling)) are two policies on the same machinery.
> [!NOTE]
> *The strong form of [§5](#5---identity-is-an-integer) still applies.* If your row has a natural key - `(suit, rank)`, `(date, ticker)`, `(species, position)` - you do not need a surrogate id. The card-game deck can be played without ids; the reference that survives is the `(suit, rank)` pair, because the data is unique by construction. Surrogate ids and generations earn their keep when the data has no natural unique tuple - which is most of the time once you start producing rows at runtime.
## Exercises
These extend the §5 deck once more, then take a step toward the simulator's variable-quantity case.
1. **Add the id column.** Add `ids = np.arange(52, dtype=np.uint32)` to your deck. Modify your sort so it reorders `ids` along with the other columns. Verify the original ids are still there, just in a new order.
2. **Find a card by id.** Implement `slot_of(ids, target)` as in the prose. Use it to look up the card with `id == 17` after a sort.
3. **Resolve the §9 bug.** With player 1 holding *ids* `[3, 17, 21, 28, 41]` (not slots), sort the deck. Use `slot_of` to translate ids to slots and print the hand. Confirm the cards are unchanged.
4. **Permutation-friendly hand query.** Rewrite `cards_held_by(locations, ids, player) -> np.ndarray` to return *ids*, not slots. The player now holds names. Test by sorting the deck after a deal and confirming `cards_held_by` still returns the same five cards.
5. **A first generation counter.** Add `gens = np.zeros(52, dtype=np.uint32)`. The 52-card deck does not actually recycle, but extend a small `swap_remove`-like operation: pop the last card from the deck (location 0), insert a "fresh" card at the freed slot, and bump that slot's `gens` by one. Take a `CreatureRef`-style `(id, gen)` reference *before* the operation. After the operation, look up the slot by id; check `gens[slot]` against the reference's `gen`. Confirm the dereference correctly reports stale.
6. *(stretch)* **A tiny generational arena.** Outside the deck, build a `Creatures` class with `pos: np.ndarray (float32)`, `gens: np.ndarray (uint32)`, plus `free: list[int]` of slots awaiting reuse. Implement `insert(pos) -> CreatureRef`, `remove(ref)`, and `get(ref) -> float | None`. Convince yourself by example that stale references cannot read a fresh creature's data.
7. *(stretch)* **The shape of `id_to_slot`.** Right now `slot_of` is O(N). Sketch (do not implement) the `id_to_slot` array - `np.full(N_ids, MAX, dtype=np.uint32)` - that lets you do the lookup in O(1). Note what has to happen on every reorder: when slot `i` is the new home of id `k`, `id_to_slot[k] = i`. This is a foreshadow of [§23 - Index maps](#23---index-maps). The lookup speedup costs you another column to keep aligned.
8. *(stretch)* **Compare with a real ECS handle.** Read the `Entity` documentation for [bevy_ecs](https://docs.rs/bevy_ecs/latest/bevy_ecs/entity/struct.Entity.html) (Rust) or look at the `EntityHandle` docs of any Python ECS library. Identify which of your fields and operations correspond. What does the production library add that you didn't need for the simulator? Decide consciously whether to adopt it. (This is the from-scratch-then-price-the-crate move from [§41 - Deferred abstraction](#41---deferred-abstraction) and [§42 - You can only fix what you wrote](#42---you-can-only-fix-what-you-wrote).)
Reference solutions for the deck exercises (1-5) in [10_stable_ids_and_generations_solutions.md](https://root-11.codeberg.page/intro-book-python/trunk/10_stable_ids_and_generations_solutions.html). The arena and library exercises follow the same shape and are worth working without reference.
## What's next
You now have stable references. The next thing the simulator will need is to look up a row by id in O(1) rather than O(N) - an `id_to_slot` map maintained on every reordering. That is [§23 - Index maps](#23---index-maps). It is one extra `np.ndarray`, updated whenever the columns move.
Part 2 is closed. Identity is an integer; rows align in lockstep; SoA is the default; the singleton drops out; sort breaks indices and ids fix it. The next phase is *Time & passes*, starting with [§11 - The tick](#11---the-tick). The ecosystem simulator from `code/sim/SPEC.md` is about to start running.
# 11 - The tick
A program's life has a shape:
- **Start-up** - initialisation. Tables are allocated, inputs are opened, the RNG is seeded, the w