{"id":51820324,"url":"https://github.com/bbalouki/soatable","last_synced_at":"2026-07-22T04:43:00.339Z","repository":{"id":367812581,"uuid":"1274210190","full_name":"bbalouki/soatable","owner":"bbalouki","description":"A C++23 Structure-of-Arrays (SoA) table with sparse columns and generational handles for data-oriented applications.","archived":false,"fork":false,"pushed_at":"2026-07-04T21:33:16.000Z","size":317,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-22T04:42:51.563Z","etag":null,"topics":["cache-friendly","columnar","cpp","cpp23","cpp26","data-oriented-design","ecs","game-engine","header-only","high-performance","quantization","soa","structure-of-arrays","table"],"latest_commit_sha":null,"homepage":"https://github.com/bbalouki/soatable","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bbalouki.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-19T09:32:23.000Z","updated_at":"2026-07-04T21:33:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/bbalouki/soatable","commit_stats":null,"previous_names":["bbalouki/soatable"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bbalouki/soatable","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbalouki%2Fsoatable","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbalouki%2Fsoatable/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbalouki%2Fsoatable/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbalouki%2Fsoatable/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bbalouki","download_url":"https://codeload.github.com/bbalouki/soatable/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbalouki%2Fsoatable/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35748424,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-22T02:00:06.236Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache-friendly","columnar","cpp","cpp23","cpp26","data-oriented-design","ecs","game-engine","header-only","high-performance","quantization","soa","structure-of-arrays","table"],"created_at":"2026-07-22T04:42:59.589Z","updated_at":"2026-07-22T04:43:00.313Z","avatar_url":"https://github.com/bbalouki.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SoaTable\n\n[![Build](https://github.com/bbalouki/soatable/actions/workflows/ci.yaml/badge.svg)](https://github.com/bbalouki/soatable/actions/workflows/ci.yaml)\n![C++23](https://img.shields.io/badge/C%2B%2B-23-blue.svg)\n![CMake](https://img.shields.io/badge/CMake-3.25%2B-green.svg)\n![License MIT](https://img.shields.io/badge/License-MIT-yellow.svg)\n![Header-only](https://img.shields.io/badge/Header--only-Yes-blue)\n![Zero dependencies](https://img.shields.io/badge/Core%20dependencies-None-brightgreen)\n\n**SoaTable is a header-only, dependency-free C++23 Structure-of-Arrays (SoA) library for\ndata-oriented design: a cache-friendly columnar data table with sparse optional columns, stable\ngenerational handles, and a familiar row-based API.** It is built for high-performance workloads\nsuch as game engines and Entity-Component-System (ECS) worlds, quantitative-finance tick and\nportfolio stores, and aerospace telemetry and time-series analytics, delivering column-oriented\nspeed without giving up the way you naturally think about rows and records.\n\n**SoaTable stores tables of data so that the computer can read\nthem as fast as physically possible.** \n\nYou still think in terms of rows, \"here is one trade\", \"here\nis one spaceship\", \"here is one game character\", but underneath, SoaTable stores the data in the\nshape the hardware actually prefers. The result is a familiar, row-shaped way of working that runs\ndramatically faster on the kinds of questions real programs ask.\n\nThis document is written to be read start to finish, by anyone. You do not need to open a single\nsource file to understand what SoaTable does, why it is built the way it is, or how to use every one\nof its features. Where code appears, the paragraph above it already explains the idea in plain\nwords, so you can read the prose and skip the code if you like.\n\n## Table of contents\n\n- [Start here: the problem, in plain words](#start-here-the-problem-in-plain-words)\n- [The mental model](#the-mental-model)\n- [The philosophy](#the-philosophy)\n- [Is SoaTable the right tool for you?](#is-soatable-the-right-tool-for-you)\n- [Quick start](#quick-start)\n- [The guided tour](#the-guided-tour)\n  - [1. The table: rows, handles, and sparse columns](#1-the-table-rows-handles-and-sparse-columns)\n  - [2. Reading values safely](#2-reading-values-safely)\n  - [3. Asking questions: views and selects](#3-asking-questions-views-and-selects)\n  - [4. A friendlier handle: `row_handle`](#4-a-friendlier-handle-row_handle)\n  - [5. Sorting and bulk loading](#5-sorting-and-bulk-loading)\n  - [6. Zero-copy columns and validity bitmaps](#6-zero-copy-columns-and-validity-bitmaps)\n  - [7. Compact storage helpers](#7-compact-storage-helpers)\n  - [8. The compute layer: math over columns](#8-the-compute-layer-math-over-columns)\n  - [9. Queries and group-by](#9-queries-and-group-by)\n  - [10. Choosing how memory is laid out](#10-choosing-how-memory-is-laid-out)\n  - [11. Sharing a table across threads](#11-sharing-a-table-across-threads)\n  - [12. Time-series helpers](#12-time-series-helpers)\n  - [13. Units that catch mistakes at compile time](#13-units-that-catch-mistakes-at-compile-time)\n  - [14. Runtime schemas: the dynamic table](#14-runtime-schemas-the-dynamic-table)\n  - [15. Saving and loading](#15-saving-and-loading)\n  - [16. Building a table from a struct](#16-building-a-table-from-a-struct)\n  - [17. Running without exceptions](#17-running-without-exceptions)\n- [The opt-in headers at a glance](#the-opt-in-headers-at-a-glance)\n- [Domain cookbooks](#domain-cookbooks)\n- [Installation](#installation)\n- [Benchmarks: what the speed actually is](#benchmarks-what-the-speed-actually-is)\n- [Building, testing, and docs](#building-testing-and-docs)\n- [Versioning and naming](#versioning-and-naming)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Start here: the problem, in plain words\n\nPicture a filing cabinet where every drawer holds folders, and each folder is one record, one\nperson, one trade, one sensor reading. To find the average age of everyone in the cabinet, you have\nto pull out every single folder, flip past the name, the address, the phone number, and finally read\nthe one line you care about. You touched everything to use almost nothing.\n\nComputers have exactly this problem, and it is the single biggest reason programs run slower than\nthey should. When a program reads even one number from memory, the hardware does not fetch just that\nnumber, it fetches a whole **cache line**, a fixed-size chunk of nearby bytes, on the bet that you\nwill want the neighbours too. If your records are stored as whole folders sitting next to each other\n(the normal way, called **Array of Structures**, or AoS), then reading one field drags all the\nother fields of that record along for the ride, wasting most of every fetch.\n\n```cpp\n// The normal layout: each record is a folder; folders sit back to back.\nstruct Particle { double x, y, vx, vy; std::string name; };\nstd::vector\u003cParticle\u003e particles;   // x,y,vx,vy,name | x,y,vx,vy,name | ...\n```\n\nSoaTable turns the cabinet on its side. Instead of one drawer per record, it keeps **one drawer per\nfield**: all the names together in one drawer, all the ages together in another, all the prices in a\nthird. This is called a **Structure of Arrays**, or SoA. Now \"average age\" reads a single tight\ndrawer end to end, and every byte the hardware fetches is one you actually wanted.\n\n```text\n  Array of Structures (folders)        Structure of Arrays (SoaTable)\n  -----------------------------        ------------------------------\n  [name age city][name age city] ...   names:  [name][name][name] ...\n  [name age city][name age city] ...   ages:   [age ][age ][age ] ...\n                                        cities: [city][city][city] ...\n```\n\nThat one change is worth a lot, but SoaTable goes further with two refinements that matter in real\nprograms:\n\n- **The drawers are independent and sparse.** Not every record has every field. A free game\n  character might have a position but no weapon; a trade might have a price but no settlement note.\n  In SoaTable, a field's drawer only stores the records that actually have that field, so you never\n  pay memory for the blanks.\n- **Every record keeps a stable ticket.** When you add a record you get back a small, permanent\n  **handle**. You can delete records, add new ones, and re-sort the whole table, and your handle\n  still points at the right record, or correctly tells you it is gone. No dangling indices, no\n  surprises.\n\n## The mental model\n\nIf you remember four ideas, you understand SoaTable:\n\n1. **A column is a field's drawer.** You declare your columns up front as small types\n   (`struct Price { double value; };`). Each becomes its own dense, tightly packed array.\n2. **A row is a record, reached by a handle.** `insert()` creates an empty record and hands you a\n   `row_id`. You then `assign` whatever fields that record happens to have.\n3. **Columns are optional per row.** A record carries only the fields you gave it. Asking for a\n   field a record does not have is a well-defined \"not present\", never a crash or garbage.\n4. **Questions start from the smallest drawer.** When you ask for \"every record that has both a\n   price and a quantity\", SoaTable looks at the smaller of the two drawers and checks each of those\n   against the other. Starting small means it examines far fewer records, which is where most of the\n   speed comes from.\n\n## The philosophy\n\nSoaTable is opinionated, and the opinions are what make it fast and trustworthy. They are worth\nstating plainly, because they explain every design choice you will meet below.\n\n- **You pay only for what you use.** The core is a single header with no dependencies. Everything\n  beyond the table itself, math, queries, on-disk storage, threading, units, lives in a separate\n  header you include _only if you reach for it_. A feature you never touch costs you nothing: not\n  memory, not binary size, not compile time.\n- **The row API never leaves you.** No matter which advanced storage or layout you pick underneath,\n  the way you insert, assign, read, and query stays identical. You choose performance trade-offs\n  without rewriting your program.\n- **Predictability over cleverness.** SoaTable is used in places where a surprise is unacceptable,\n  trading systems that cannot stall, flight software that cannot throw. So it offers exception-free\n  error paths, deterministic memory strategies, and a build mode with no exceptions at all. When a\n  clever shortcut and a predictable one disagree, predictability wins.\n- **Stable identity is sacred.** Handles survive deletion, insertion, and sorting. A stale handle is\n  _detected_, never silently pointed at the wrong record.\n\n## Is SoaTable the right tool for you?\n\n**Reach for it when:**\n\n- **Games and simulations (Entity-Component-System worlds).** Thousands of entities, each with a\n  sparse mix of components, stable identity across frames, and systems that walk a few components at\n  a time.\n- **Finance.** Tick and portfolio tables, columnar analytics, group-by aggregation, compact storage,\n  and fast snapshots.\n- **Engineering and aerospace telemetry.** Frame stores with optional channels, smoothly changing\n  signals, recompute-only-what-changed workflows, unit safety, and a no-exceptions build for flight\n  software.\n- **Scientific and data work.** Hand a whole column straight to a math kernel or a numerical library\n  with no copying, run vectorized operations, or memory-map columns that are larger than RAM.\n\n**Prefer a plain `std::vector\u003cYourStruct\u003e` when** the table is tiny, every record always has every\nfield, and every pass over the data touches every field anyway. SoaTable's advantage grows with\nsize, with sparsity, and with how selective your questions are.\n\n## Quick start\n\nHere is a complete, runnable program. Read the comments and you have the whole core idea in your\nhands.\n\n```cpp\n#include \u003csoatable/soatable.hpp\u003e\n#include \u003cprint\u003e\n\n// Columns are tiny named types. The name is what makes a column distinct.\nstruct Health { float value; };\nstruct Name   { std::string value; };\nstruct Poisoned {};                 // A field with no data: just a yes/no tag.\n\nint main() {\n    soatable::soa_table\u003cHealth, Name, Poisoned\u003e table;\n\n    const auto id = table.insert();      // Create a record; get back a stable handle.\n    table.assign\u003cName\u003e(id, \"Warrior\");   // This record has a Name ...\n    table.assign\u003cHealth\u003e(id, 100.0f);    // ... and a Health, but no Poisoned tag.\n\n    // Walk every record that has both Health and Name. The structured binding\n    // gives you real references: writing through them edits the table directly.\n    for (auto [row, health, name] : table.view\u003cHealth, Name\u003e()) {\n        health.value -= 10.0f;\n        std::println(\"{} has {} health\", name.value, health.value);\n    }\n\n    // Optional columns: include Poisoned when present, without requiring it.\n    for (auto [row, name, poison] : table.select\u003cName, std::optional\u003cPoisoned\u003e\u003e()) {\n        if (poison) std::println(\"{} is poisoned!\", name.get().value);\n    }\n}\n```\n\n## The guided tour\n\nEach section below opens with the everyday idea, then shows it in code. Together they cover every\nfeature SoaTable offers.\n\n### 1. The table: rows, handles, and sparse columns\n\n**The idea.** A table is a fixed set of columns. You add records with `insert()`, attach fields with\n`assign`, drop a field with `unassign`, and remove a record with `erase`. The handle you get back\n(`row_id`) is a _generational_ ticket: it remembers not just _where_ a record lives but _which_\nrecord, so if a slot is reused by a later record, your old handle correctly reports itself invalid\ninstead of silently pointing at the newcomer. (This defeats a classic bug known as the ABA problem.)\n\n```cpp\nsoatable::soa_table\u003cName, Age\u003e table;\n\nconst auto id = table.insert();\ntable.assign\u003cAge\u003e(id, Age{30});\ntable.contains\u003cAge\u003e(id);      // true: this record has an Age\ntable.unassign\u003cAge\u003e(id);      // drop just that field\ntable.erase(id);              // remove the whole record\ntable.is_valid(id);           // false: the handle knows its record is gone\n```\n\nA column with no data members (like `Poisoned` above) is a **tag**: it answers a yes/no question and\ncosts almost nothing. This is the natural way to mark state, \"is frozen\", \"is poisoned\", \"needs\nreview\", without inventing a boolean field.\n\n### 2. Reading values safely\n\n**The idea.** Different situations want different answers to \"what if the field isn't there?\".\nSoaTable gives you three readers so you can pick the failure mode that fits, rather than forcing one\non you.\n\n| Reader                | When the field is missing                | Use it when                             |\n| --------------------- | ---------------------------------------- | --------------------------------------- |\n| `get\u003cT\u003e(id)`          | throws an exception                      | you are confident it is there           |\n| `try_get\u003cT\u003e(id)`      | returns `nullptr`                        | you want a cheap \"is it there?\" check   |\n| `get_expected\u003cT\u003e(id)` | returns a `std::expected` value-or-error | hot paths, or builds with no exceptions |\n\n```cpp\nif (auto* hp = table.try_get\u003cHealth\u003e(id)) {   // no exception, just a pointer check\n    hp-\u003evalue -= 5.0f;\n}\n```\n\nReading through a `const` table gives you `const` references all the way down, so a table is safe to\nshare read-only across a program without anyone accidentally mutating it.\n\n### 3. Asking questions: views and selects\n\n**The idea.** Most real work is \"for every record that has _these_ fields, do something\". That is\nwhat `view` and `select` are for. Both walk only the records that have _all_ the requested fields,\nand both start from the smallest of those columns so they examine as few records as possible.\n\n- **`view\u003cCols...\u003e()`** is the everyday workhorse. Its structured bindings hand you **real\n  references**, so you read and write fields directly with no ceremony.\n- **`select\u003cCols...\u003e()`** is the more flexible form. It is the one that understands\n  `std::optional\u003cT\u003e`, letting a column be included _when present_ without being required.\n\n```cpp\n// Move every entity that has both a Position and a Velocity.\nfor (auto [id, pos, vel] : world.view\u003cPosition, Velocity\u003e()) {\n    pos.x += vel.x;\n    pos.y += vel.y;\n}\n\n// Include Age when the record has it; the record is still listed if it doesn't.\nfor (auto [id, name, age] : table.select\u003cName, std::optional\u003cAge\u003e\u003e()) {\n    if (age) { /* this record has an Age */ }\n}\n```\n\nWith `select`, fields come wrapped so you read them with `.get()`; with `view`, the binding is the\nreference itself.\n\n### 4. A friendlier handle: `row_handle`\n\n**The idea.** Carrying a `row_id` plus the table it belongs to everywhere can be tedious.\n`row_handle` bundles the two together so you can act on a record directly, and a default-constructed\nhandle is safely \"unbound\" (it reports itself invalid rather than misbehaving).\n\n```cpp\nsoatable::row_handle\u003cName, Age\u003e handle{ table.insert(), table };\nhandle.assign\u003cName\u003e(\"Dora\");\nhandle.assign\u003cAge\u003e(40);\nhandle.get\u003cName\u003e().value;     // \"Dora\"\nhandle.contains\u003cAge\u003e();       // true\nhandle.erase();               // remove the record through the handle\n```\n\n### 5. Sorting and bulk loading\n\n**The idea.** Sometimes order matters, ticks by timestamp, players by score. SoaTable can physically\nreorder every column together so the data stays meaningfully ordered (which later lets time-series\nwindows and column views be correct). And when you are loading lots of records at once, batch\noperations avoid the per-record overhead.\n\n```cpp\n// Sort by one column.\ntable.sort_by_column\u003cAge\u003e([](const Age\u0026 a, const Age\u0026 b) { return a.value \u003c b.value; });\n\n// Sort by a primary key, breaking ties with a secondary key.\ntable.sort_by_multi(\n    std::pair\u003cName, std::function\u003cbool(const Name\u0026, const Name\u0026)\u003e\u003e{\n        {}, [](const Name\u0026 a, const Name\u0026 b) { return a.value \u003c b.value; } },\n    std::pair\u003cAge, std::function\u003cbool(const Age\u0026, const Age\u0026)\u003e\u003e{\n        {}, [](const Age\u0026 a, const Age\u0026 b) { return a.value \u003c b.value; } });\n\n// Sort in parallel above a size threshold (serial below it, where threads would not pay off).\ntable.sort_by_column_parallel\u003cAge\u003e([](const Age\u0026 a, const Age\u0026 b) { return a.value \u003c b.value; });\n\n// Bulk-load many records, then fill a column from a sequence.\nconst auto ids = table.insert_batch(3);\nstd::array\u003cAge, 3\u003e ages{ Age{20}, Age{30}, Age{40} };\ntable.assign_batch\u003cAge\u003e(ids, ages.begin());\n```\n\n### 6. Zero-copy columns and validity bitmaps\n\n**The idea.** Because each column is one dense, contiguous, aligned array, SoaTable can hand it to\nyou as a `std::span`, a borrowed window over the real storage, with **no copying**. That column can\ngo straight into a SIMD kernel, an FFT, a linear-algebra library, or any code that wants a plain\narray of numbers. Alongside it, a **validity bitmap** is a compact yes/no record of which rows have\nthe column, perfect for fast \"how many\", \"which ones\", and branch-free masked loops. (It is also the\nnatural bridge to the Apache Arrow data format.)\n\n```cpp\nstd::span\u003cconst Price\u003e prices = table.column\u003cPrice\u003e();   // borrowed, aligned, no copy\nconst auto valid = table.validity\u003cPrice\u003e();              // packed presence bitmap\nstd::println(\"{} prices, {} rows actually priced\", prices.size(), valid.count());\n```\n\n### 7. Compact storage helpers\n\n**The idea.** Some data does not need a full-size number. A confidence score between 0 and 1, a few\nboolean flags, a value that only ever drifts a little, these can be squeezed into far fewer bytes,\nwhich means more of them fit in cache and everything runs faster. These helpers live in the core\nheader and can be used directly as column types.\n\n- **`quantized_float`** stores a floating-point value, within a known range, in a small integer (for\n  example 8 bits), trading a little precision for a lot of space.\n- **`packed_bits`** packs several small fields into the bits of one integer.\n- **`delta_value`** tracks a value that changes gently by storing scaled _deltas_ rather than full\n  numbers, ideal for smooth signals like altitude.\n- **`dirty_mask`** is a set of on/off flags, the clean way to record \"which parts of this record\n  changed\" so you can recompute only those.\n\n```cpp\n// A confidence in [0, 1] kept in a single byte.\nusing Confidence = soatable::quantized_float\u003cstd::uint8_t, 0, 1000, 8\u003e;\nConfidence c{0.75};\ndouble back = c.get();        // ~0.75, dequantized on read\n\n// Smoothly varying altitude tracked as deltas.\nsoatable::delta_value\u003cfloat\u003e altitude{0.0F};\naltitude.apply_delta(altitude.get_delta(101.0F));   // nudge toward 101 m\nfloat metres = altitude.get();\n\n// Per-record flags.\nenum class Subsystem : std::uint32_t { avionics = 1, propulsion = 2, thermal = 4 };\nsoatable::dirty_mask\u003cSubsystem\u003e flags;\nflags.mark_dirty(Subsystem::thermal);\nbool changed = flags.is_dirty(Subsystem::thermal);  // true\n```\n\n### 8. The compute layer: math over columns\n\n**The idea.** Once your data is in columns, doing math over it should be one line, and it should be\nfast because the compiler can vectorize a tight loop over an aligned array. Include\n`\u003csoatable/compute.hpp\u003e` for that. It covers single-column transforms and reductions, scalar\nbroadcasting, masked and strided variants, chunk-parallel versions, and, crucially, **cross-column**\nmath like `pnl = price * qty` that walks the join of the inputs and only touches records that have\nall of them.\n\n```cpp\n#include \u003csoatable/compute.hpp\u003e\n\n// notional = price * qty, computed for every record that has both.\nsoatable::compute::assign_from\u003cNotional, Price, Qty\u003e(\n    book, [](const Price\u0026 p, const Qty\u0026 q) { return Notional{ p.value * q.value }; });\n\n// Sum a column with a reduction.\ndouble gross = soatable::compute::reduce_column\u003cNotional\u003e(\n    book, 0.0, [](double acc, const Notional\u0026 n) { return acc + n.value; });\n```\n\n### 9. Queries and group-by\n\n**The idea.** Analytics ergonomics, \"filter, then group, then total\", without dragging in a database\nor query engine. Include `\u003csoatable/query.hpp\u003e`. `select_where` filters the join by any condition you\nwrite; `group_sum`, `group_count`, and `group_reduce` aggregate records into buckets keyed by\nwhatever you project out of a column. All of it composes on top of the smallest-drawer scan, so it\nstays fast.\n\n```cpp\n#include \u003csoatable/query.hpp\u003e\n\n// Total notional volume per symbol.\nauto volume = soatable::query::group_sum\u003cSymbol, Notional\u003e(\n    ticks,\n    [](const Symbol\u0026 s){ return s.value; },     // group key\n    [](const Notional\u0026 n){ return n.value; });   // value to sum\n\n// Only the large trades.\nfor (auto row : soatable::query::select_where\u003cSymbol, Notional\u003e(\n        ticks, [](auto trade){ return trade.template get\u003cNotional\u003e().value \u003e 10000.0; })) {\n    // row.template get\u003cSymbol\u003e().value ...\n}\n```\n\n### 10. Choosing how memory is laid out\n\n**The idea.** The default layout, one flat, contiguous, SIMD-aligned array per column, is best for\nanalytics. But other workloads want other trade-offs, and SoaTable lets you pick the layout _without\nchanging a line of your row code_. The choice is part of the table's type:\n\n- **`soa_table\u003c...\u003e`** (default): flat, contiguous, 64-byte aligned. Best for scanning and number\n  crunching.\n- **`aosoa_table\u003cTile, ...\u003e` / `chunked_soa_table\u003cChunk, ...\u003e`**: storage split into fixed-size tiles\n  or chunks, so growth never has to copy the existing data. Good for streaming and bounded latency.\n  `column_tiles\u003cT\u003e()` views the data as per-tile aligned spans.\n- **`custom_soa_table\u003cAllocator, ...\u003e`**: bring your own allocator. The opt-in `\u003csoatable/pmr.hpp\u003e`\n  gives you `pmr_soa_table`, which draws memory from a `std::pmr` arena, monotonic buffer, or pool,\n  for deterministic, fragmentation-free allocation in real-time code.\n- **`\u003csoatable/mmap.hpp\u003e` `mmap_soa_table`**: column storage is memory-mapped and paged in by the\n  operating system on demand, so a single column can be far larger than physical RAM.\n\n```cpp\n// Tiled storage in chunks of 4; growth never copies existing chunks.\nsoatable::aosoa_table\u003c4, Price\u003e tiled;\nfor (int i = 0; i \u003c 10; ++i)\n    tiled.assign\u003cPrice\u003e(tiled.insert(), Price{ static_cast\u003cdouble\u003e(i) });\nauto chunks = tiled.column_tiles\u003cPrice\u003e();   // per-tile aligned spans\n```\n\n### 11. Sharing a table across threads\n\n**The idea.** A common pattern is \"many readers, one writer\", lots of worker threads reading a market\ndata table while one thread ingests updates. `\u003csoatable/concurrent.hpp\u003e` wraps any table in a\n`synchronized_table` that lets many readers in at once but gives writers exclusive access, all\nthrough scoped callbacks so the locking is automatic and correct.\n\n```cpp\n#include \u003csoatable/concurrent.hpp\u003e\n\nsoatable::synchronized_table\u003cTable\u003e shared;\nshared.write([](Table\u0026 t){ t.insert(); });             // exclusive\nauto n = shared.read([](const Table\u0026 t){ return t.size(); });  // shared with other readers\n```\n\n### 12. Time-series helpers\n\n**The idea.** When a column is ordered by time, you often want rolling windows and differences, a\n20-tick moving average, the change since the last reading, and you want to recompute only the rows\nthat actually changed. `\u003csoatable/timeseries.hpp\u003e` provides exactly these, operating on a column's\ndense order (so sort by your time key first).\n\n```cpp\n#include \u003csoatable/timeseries.hpp\u003e\n\n// 2-period rolling mean of the Price column, in its stored order.\nauto means = soatable::timeseries::rolling_mean_column\u003cPrice\u003e(\n    book, [](const Price\u0026 p){ return p.value; }, 2);\n\n// Visit only the records flagged changed, and clear the flags as you go.\nusing Status = soatable::dirty_mask\u003cSubsystem\u003e;   // a dirty_mask used directly as a column\nsoatable::timeseries::for_each_dirty\u003cStatus\u003e(frames, [](soatable::row_id, Status\u0026 s){\n    // recompute this record, then s.reset();\n});\n```\n\nThere are also free-function forms (`rolling_sum`, `rolling_mean`, `deltas`) that work on any span of\nvalues.\n\n### 13. Units that catch mistakes at compile time\n\n**The idea.** Adding metres to seconds is always a bug, so it should never compile. `\u003csoatable/units.hpp\u003e`\ngives you `quantity\u003cT, Dimension\u003e` types where multiplication and division combine units correctly\n(distance / time gives a velocity) while adding mismatched units is a **compile error**. These can be\nused directly as column types, so unit safety follows your data into the table.\n\n```cpp\n#include \u003csoatable/units.hpp\u003e\nnamespace un = soatable::units;\n\nauto speed = un::length\u003c\u003e{100.0} / un::duration\u003c\u003e{4.0};  // a velocity, by construction\ndouble v = speed.value();                                 // 25.0\n// un::length\u003c\u003e{1.0} + un::duration\u003c\u003e{1.0};   // \u003c-- would not compile, on purpose\n```\n\n### 14. Runtime schemas: the dynamic table\n\n**The idea.** Sometimes you do not know the columns until the program is running, loading a CSV whose\nheaders you have not seen, or letting a user define fields. `\u003csoatable/dynamic.hpp\u003e` gives a\n`dynamic_table` whose columns are added and removed by name at runtime, type-checked on every access\n(without RTTI), and each column can carry string metadata such as a unit label.\n\n```cpp\n#include \u003csoatable/dynamic.hpp\u003e\n\nsoatable::dynamic_table dyn;\ndyn.add_column\u003cdouble\u003e(\"altitude\");\ndyn.set_metadata(\"altitude\", \"unit\", \"metres\");\n\nconst auto row = dyn.insert_row();\ndyn.set\u003cdouble\u003e(row, \"altitude\", 1280.0);\ndouble a = *dyn.get\u003cdouble\u003e(row, \"altitude\");   // 1280.0\n```\n\n### 15. Saving and loading\n\n**The idea.** To checkpoint a simulation or snapshot a tick store, you want to write the whole table\nto bytes and read it back exactly, handles, generations, and column order included.\n`\u003csoatable/serialize.hpp\u003e` does this for tables of plain, trivially-copyable columns, wrapping the\ndata in a versioned, schema-checked header so a load can _reject_ a buffer that does not match,\nrather than silently corrupting your data. (The format is native byte order, meant for snapshots read\nback on the same kind of machine.)\n\n```cpp\n#include \u003csoatable/serialize.hpp\u003e\n\nauto bytes = soatable::save(pod_table);                 // -\u003e std::vector\u003cstd::byte\u003e\nsoatable::soa_table\u003cPrice, Qty\u003e restored;\nauto status = soatable::load(restored, bytes);\nbool ok = (status == soatable::serialize_status::ok);   // also: bad_magic, schema_mismatch, ...\n```\n\n### 16. Building a table from a struct\n\n**The idea.** You may already have a plain struct describing a record and would rather not list its\nfields twice. `\u003csoatable/reflect.hpp\u003e` lets you derive the table type from the struct. Until C++26\nreflection ships in compilers, you write a single one-line mapping; once a compiler supports\nreflection, even that line becomes unnecessary and the mapping is automatic.\n\n```cpp\n#include \u003csoatable/reflect.hpp\u003e\n\nstruct ParticleSchema {};   // a tag naming your schema\ntemplate \u003c\u003e struct soatable::columns_of\u003cParticleSchema\u003e\n    : soatable::column_list\u003cPosition, Velocity\u003e {};\n\nusing ParticleTable = soatable::table_for\u003cParticleSchema\u003e;   // == soa_table\u003cPosition, Velocity\u003e\n```\n\n### 17. Running without exceptions\n\n**The idea.** Embedded, kernel, and flight targets often forbid C++ exceptions entirely. Define\n`SOATABLE_NO_EXCEPTIONS` and every place that would `throw` instead calls a terminating handler; you\nlean on the non-throwing readers (`try_get`, `get_expected`) for normal error handling. SoaTable uses\nno RTTI either, and a dedicated CI leg builds the whole library with `-fno-exceptions` to keep this\npath honest.\n\n## The opt-in headers at a glance\n\nThe core, `soatable/soatable.hpp`, gives you the table, spans, validity bitmaps, and the compression\nhelpers, with no extra dependencies. Everything else is a header you add only when you want it.\n\n| Header                    | What it adds                                      | Extra standard headers it pulls in |\n| ------------------------- | ------------------------------------------------- | ---------------------------------- |\n| `soatable/soatable.hpp`   | the table, spans, validity, compression helpers   | nothing extra                      |\n| `soatable/compute.hpp`    | column math, broadcasting, chunk-parallel compute | `\u003cfuture\u003e`, `\u003cthread\u003e`             |\n| `soatable/query.hpp`      | `select_where`, group-by aggregation              | `\u003cunordered_map\u003e`                  |\n| `soatable/serialize.hpp`  | versioned `save()` / `load()`                     | nothing extra                      |\n| `soatable/pmr.hpp`        | `pmr_soa_table` (arena / pool allocation)         | `\u003cmemory_resource\u003e`                |\n| `soatable/mmap.hpp`       | `mmap_soa_table` (larger-than-RAM columns)        | `\u003csys/mman.h\u003e` / `\u003cwindows.h\u003e`     |\n| `soatable/concurrent.hpp` | `synchronized_table` (many readers, one writer)   | `\u003cshared_mutex\u003e`                   |\n| `soatable/timeseries.hpp` | rolling windows, deltas, dirty scans              | nothing extra                      |\n| `soatable/units.hpp`      | dimensional `quantity` types                      | nothing extra                      |\n| `soatable/dynamic.hpp`    | runtime `dynamic_table`                           | `\u003cunordered_map\u003e`                  |\n| `soatable/reflect.hpp`    | `table_for\u003cStruct\u003e`                               | nothing extra                      |\n\n## Domain cookbooks\n\nThese are complete, runnable programs (built as `soatable_cookbook_*` targets) that put the pieces\ntogether for a real domain:\n\n- [Finance, a tick store](example/cookbook/finance_tick_store.cpp): ingest trades, derive P\u0026L per\n  record, total notional volume per symbol, and filter the large trades.\n- [Game / ECS](example/cookbook/game_ecs.cpp): a sparse-component world with a motion system over\n  `view\u003cPosition, Velocity\u003e()` and a data-less tag for frozen entities.\n- [Aerospace, telemetry](example/cookbook/aerospace_telemetry.cpp): `delta_value` altitude tracking,\n  `dirty_mask` per-frame flags, and a validity bitmap over temperature readings.\n- [Scientific data](example/cookbook/scientific_table.cpp): zero-copy span compute and\n  `quantized_float` storage.\n\nA broader [feature tour](example/soatable_example.cpp) exercises most headers in one program.\n\n## Installation\n\nSoaTable is header-only and needs a C++23 compiler (GCC 13+, Clang 18+, or MSVC from VS 2022) and\nCMake 3.25+.\n\n### CMake\n\n```cmake\nfind_package(soatable CONFIG REQUIRED)\ntarget_link_libraries(your_target PRIVATE soatable::soatable)\n```\n\n### Conan / vcpkg\n\n```sh\nconan create .                                                   # header-only recipe at the repo root\nvcpkg install soatable --overlay-ports=packaging/vcpkg/ports     # overlay port\n```\n\n### Single-header drop-in\n\nThe core `soatable.hpp` is self-contained. To emit a stamped, vendorable copy:\n\n```sh\npython scripts/amalgamate.py            # writes dist/soatable.hpp\n```\n\nThe opt-in add-on headers are kept separate on purpose, so you pay only for what you include. Pass\n`--with compute,query` to bundle specific ones next to the core.\n\n### Debugger visualizers\n\nSoaTable ships pretty-printers that render its types (tables, handles, `delta_value`,\n`quantized_float`, ...) as readable summaries in a debugger. When you consume the library through\nCMake, the Visual Studio (`.natvis`) visualizer is embedded into your program automatically, with no\nsetup. The GDB and LLDB printers are opt-in: follow the short setup in\n[Visualizers](tools/visualizers/README.md).\n\n## Benchmarks: what the speed actually is\n\nThe whole point of SoaTable is speed on selective questions, so the claim is measured, not asserted.\nThe suite covers insertion, erase churn, single-column and parallel sort, sparse selection across a\nsweep of densities, the smallest-drawer heuristic, SoA-versus-tiled selection, and compute\nthroughput, each against an Array-of-Structures baseline and a hand-written columnar baseline.\n\n```sh\n./scripts/run_benchmarks.sh             # builds the `bench` preset, writes bench/results.json\n```\n\nA representative result, a selective join over 250,000 rows in a Release build:\n\n| Method                                            | Time          |\n| ------------------------------------------------- | ------------- |\n| SoaTable `select` (starts from smallest drawer)   | ~168,000 ns   |\n| Same, but forced to start from the largest drawer | ~1,547,000 ns |\n| Hand-rolled columnar scan                         | ~1,058,000 ns |\n| Array-of-Structures branch scan                   | ~1,303,000 ns |\n\nThe roughly 9x gap between starting from the smallest drawer and being forced to start from the\nlargest is the whole thesis of the library in one number: examine fewer records, finish sooner.\nNumbers are machine dependent; reproduce them with the harness above.\n\n## Building, testing, and docs\n\n```sh\ncmake --preset clang        # or: gcc, msvc, asan-ubsan, bench\ncmake --build --preset clang\nctest --preset clang\n```\n\nCI builds GCC 13, Clang 18, and MSVC, plus an AddressSanitizer + UndefinedBehaviorSanitizer leg, all\nthrough these same presets, so a green local run should match CI. Build the Doxygen API site locally\nwith:\n\n```sh\ncmake -S . -B build-docs -DSOATABLE_BUILD_DOCUMENTATION=ON\ncmake --build build-docs --target docs\n```\n\n## Versioning and naming\n\nSoaTable follows [Semantic Versioning](https://semver.org). The public API uses std-style lowercase\nnames (`soa_table`, `row_handle`, `column_vector`, `delta_value`, `dirty_mask`); the earlier\nPascalCase spellings remain as `[[deprecated]]` aliases for one migration window so existing code\nkeeps compiling while you move.\n\n## Contributing\n\nContributions are welcome. [CONTRIBUTING](CONTRIBUTING.md) explains the philosophy behind the\nproject and how to make a change that fits it, the principles, the local build and sanitizer\nworkflow, and what a good pull request looks like.\n\n## License\n\nMIT, see [LICENSE](LICENSE).\n\n## Keywords\n\nC++ Structure-of-Arrays (SoA) library, data-oriented design (DOD), header-only C++23 library,\ncache-friendly columnar data table, sparse columns, Entity-Component-System (ECS) storage, ECS\nlibrary for games, quantitative finance tick store, columnar analytics and group-by aggregation,\ntime-series rolling windows, zero-copy `std::span` columns, SIMD-friendly aligned storage, Apache\nArrow-style validity bitmaps, generational handles / handle-slot map, memory-mapped larger-than-RAM\ncolumns, PMR arena allocator table, dimensional analysis / units, no-exceptions and no-RTTI build\nfor embedded and flight software, CMake / Conan / vcpkg package. A fast, modern C++ alternative to a\nplain `std::vector\u003cStruct\u003e` (Array-of-Structures / AoS) for high-performance data processing.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbalouki%2Fsoatable","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbbalouki%2Fsoatable","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbalouki%2Fsoatable/lists"}