cpu-performance-engineering
A reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.
https://github.com/usamahz/cpu-performance-engineering
Last synced: 29 days ago
JSON representation
10. NUMA and multi-socket
Migration, balancing and measured effects
- move_pages(2) - Defines per-page migration of a running process, and a query reporting each page's node, the direct test of first touch.
- sysctl kernel numa_balancing - Defines the hinting-fault sampling behind automatic balancing and tiering, and warns the overhead may not pay off.
- Traffic Management: A Holistic Approach to Memory Placement on NUMA Systems - Proves against the kernel balancer that controller and link congestion, not remote latency, is what placement manages.
- Intel Memory Latency Checker - Measures the node-to-node latency and bandwidth matrix and loaded latency on the x86 at hand, which no datasheet states.
- High Performance Computing Tuning Guide for AMD EPYC 9004 Series Processors - Tabulates measured bandwidth by NPS mode, cores per die, boost and SMT, so the NPS trade-off is shown, not asserted.
NUMA and Linux memory placement
- NUMA (Non-Uniform Memory Access): An Overview - The one account tying first touch, policy scope, zone reclaim and page movement together from the implementer's side.
- What is NUMA? - Defines nodes, zonelists and the distance-ordered fallback that places an allocation once local memory runs out.
- NUMA Memory Policy - The normative statement of policy scopes, every mode including weighted interleave, and the cpuset intersection rule.
- Numa policy hit/miss statistics - Defines numa_hit, numa_miss and numa_foreign, the counters that show whether a policy put pages where it said.
- numactl - Reference implementation of the policy API, prints the distance table and binds a binary that cannot be rebuilt.
Topology and interconnects
- NUMA Memory Performance - Explains the firmware-rated latency and bandwidth per initiator and target, and memory-side caches, that rank nodes.
- Intel Xeon Processor Scalable Family Technical Overview - Where Intel names the mesh, UPI socket links, the directory-running home agent, and how SNC splits the cache.
- Intel Xeon 6 with P-cores Configuration and Tuning Guide for HPC Applications - Defines SNC on current parts as one node per compute die, and fixes the numactl and numastat checks of placement.
- BIOS and Workload Tuning Guide for AMD EPYC 9004 Series Processors - Discloses the I/O die, GMI and xGMI links, the NPS modes with their interleave widths, and the cache-as-NUMA override.
- Arm Neoverse CMN-700 Coherent Mesh Network Technical Reference Manual - Defines the mesh, the home nodes holding the system cache and snoop filter, and the gateways joining sockets or CXL.
11. OS and I/O
Cache and bandwidth partitioning
- Intel Resource Director Technology Architecture Specification - Defines classes of service, cache masks, bandwidth allocation and monitoring IDs, the model resctrl exposes.
- User Interface for Resource Control feature (resctrl) - Defines the filesystem through which Linux exposes Intel, AMD and Arm partitioning, and the schemata format.
- MPAM - Maps Arm's cache portion and bandwidth controls onto resctrl's schemata, and states which platform limits apply.
- CPI2: CPU performance isolation for shared compute clusters - Shows at fleet scale that cycles per instruction alone finds an interfering neighbour and the one to throttle.
- Heracles: Improving Resource Efficiency at Scale - Shows cache ways, cores, bandwidth and power must be partitioned together, or batch work reaches the tail.
Interrupts and kernel bypass
- NAPI - Defines the polling, software coalescing, busy polling and IRQ suspension knobs that trade interrupts against latency.
- DPDK Programmer's Guide - Defines the full bypass model, pinned poll-mode cores with no interrupts, that every kernel path is measured against.
- The eXpress Data Path - Measures an in-kernel programmable path against DPDK and the stack per core, with the full configuration published.
- Kernel vs. User-Level Networking: Don't Throw Out the Stack with the Interrupts - Separates direct and indirect NIC interrupt cost, measures the stack against bypass, and is where IRQ suspension began.
- AF_XDP - Defines the socket and UMEM rings handing XDP frames to user space, and the zero-copy and need-wakeup modes.
Scheduling, affinity and isolation
- EEVDF Scheduler - Defines lag and virtual deadline, which the default class schedules by, and the slice request in sched_setattr.
- The Linux Scheduler: a Decade of Wasted Cores - Proves cores sit idle while runnable threads queue, and gives the invariant checker that found the load-balancer bugs.
- Control Group v2 - Defines cpu.max throttling, cpu.weight and the cpusets that bound affinity, the controls behind every container limit.
- CPU Performance Scaling - Defines the governors, driver and boost switch that set a core's frequency, the sysfs state a measurement records.
- CPU Isolation - Ties isolcpus, nohz_full, IRQ affinity, RCU offload and cpusets into one recipe, and lists the jitter it leaves.
Syscalls and asynchronous I/O
- vdso(7) - Defines the calls the kernel answers without a mode switch, and the clocksource condition for skipping the trap.
- FlexSC: Flexible System Call Scheduling with Exception-Less System Calls - Separates a syscall's trap cost from its cache and TLB pollution, and shows the pollution can dominate.
- An Analysis of Performance Evolution of Linux's Core Operations - Measures syscall and context switch cost across kernel releases and traces each slowdown to a named mitigation.
- Efficient IO with io_uring - States the goals aio failed, and which io_uring features remove a syscall and which remove a copy.
- Understanding Modern Storage APIs: A systematic study of libaio, SPDK, and io_uring - Measures io_uring's polling modes against libaio and SPDK, and shows the kernel poller needs its own core.
12. Tail latency and production systems
Load generation and production workloads
- Open Versus Closed: A Cautionary Tale - Shows that open and closed load models disagree on response time and scheduling gains, with rules for choosing one.
- wrk2 - Issues requests on a fixed schedule and times each from when it was due, so server stalls reach the percentiles.
- Reconciling High Server Utilization and Sub-millisecond Quality-of-Service - Shows the tail, not throughput, caps a latency-critical server's utilisation, and how far co-located work lowers it.
- TailBench - Pairs latency-critical services with an open-loop harness that records sojourn against service time per request.
- Workload Analysis of a Large-Scale Key-Value Store - Measures the key, value and inter-arrival distributions of live key-value traffic, the shape load generators imitate.
Measuring the tail
- The Tail at Scale - Shows why fan-out makes a rare slow server a common slow request, and names the techniques that tolerate variance.
- Attack of the Killer Microseconds - Defines the stall band that out-of-order hardware cannot hide and a context switch cannot amortise.
- How NOT to Measure Latency - Shows that a summary without a max discards the samples that define the tail, and closed-loop load never records them.
- Coordinated Omission - The original definition of the recording error, with arithmetic for how far a reported percentile sits from the truth.
- HdrHistogram - Keeps the whole distribution at fixed relative precision in constant time, so the far percentiles and max survive.
Mechanical sympathy
- Inter Thread Latency - Measures with code the floor for handing a cache line between cores, which every queue and lock is built on.
- Single Writer Principle - States the design rule that removes write contention outright, using a contended increment's cost as the argument.
- Optimizing a Ring Buffer for Throughput - Adds cached indices to a single-producer single-consumer ring and shows with counters the coherence traffic removed.
- LMAX Disruptor - Applies the single writer rule and cache-line padding to a ring buffer, with the queue comparison that motivated it.
- Aeron - Carries the single writer and batching rules through a whole transport, the reference beyond one in-process queue.
Where jitter comes from
- rt-tests - The reference wakeup-latency measurement for Linux, whose README states that an unloaded run proves nothing.
- osnoise tracer - Counts the noise a spinning thread suffers and attributes each event to NMI, IRQ, softirq, thread or hardware.
- Tales of the Tail - Derives the queueing-ideal tail and attributes the excess to scheduling, interrupt placement, power saving and NUMA.
- Latency Implications of Virtual Memory - Measures with code the page-fault, TLB-shootdown and writeback stalls that memory mapping hides from the caller.
- The KVM halt polling system - Defines the host-side polling after a vCPU halt that trades idle host CPU for guest wakeup time, unseen by the guest.
13. Inference on CPU
GEMM and BLAS
- Anatomy of High-Performance Matrix Multiplication - Derives the cache blocking every fast CPU GEMM still uses by refining a memory model until the packed kernel falls out.
- BLIS: A Framework for Rapidly Instantiating BLAS Functionality - Reduces a BLAS to one register-tile micro-kernel per architecture and states which loop owns which level of cache.
- LLaMA Now Goes Faster on CPUs - Builds a register-tile kernel step by step and shows where outer-loop unrolling wins and where a vendor BLAS still does.
- OpenBLAS - Sets run-time dispatch for a BLAS, one kernel table per microarchitecture chosen as a DYNAMIC_ARCH build loads.
- oneDNN Matrix Multiplication Primitive - Fixes the data-type table, packed weight format and fused post-ops a CPU GEMM must expose to an inference graph.
- Scaled Dot-Product Attention (oneDNN Graph) - Fixes the fused attention pattern, f32 accumulation under bf16 inputs and the shapes the fast CPU path accepts.
Matrix extensions
- Using XSTATE features in user space applications - Defines the arch_prctl permission Linux requires for AMX tile data and the trap a tile instruction takes until granted.
- Add Intel Advanced Matrix Extensions (AMX) support to ggml - Holds the design report for the ggml AMX path, whose double buffering hides applying scales between tile products.
- SME Programmer's Guide - Walks through SME2 int8 and f32 matmul and gemv kernels and a lookup-table path, on a unit so far only in client parts.
Quantization
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference - Defines the affine scale and zero-point scheme with integer accumulation that every int8 CPU runtime still implements.
- Nuances of int8 Computations - States where int8 saturates on pre-VNNI x86, the compensation that makes signed by signed work, and what VNNI removes.
- Quantize ONNX models - Defines the QDQ form against QOperator and dynamic against static, and which sign choices are safe on which CPUs.
- k-quants - Defines super-block formats with quantized scales and the perplexity against size curve behind mixing types per tensor.
- T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edge - Replaces unpacking and multiplying low-bit weights with bit-sliced lookups, so a narrower weight costs less to apply.
Runtimes
- oneDNN - Generates VNNI and AMX kernels at run time under PyTorch, TensorFlow and OpenVINO, and calls Compute Library on Arm.
- ggml - Defines the quantized block formats and the per-ISA dot-product kernels over them that every llama.cpp type rests on.
- llama.cpp - Where new quantization types, kernels and thread pools land first, each with the perplexity and speed table behind it.
- ONNX Runtime MLAS - Holds the CPU provider's GEMM, int8 and int4 MatMul kernels, dispatched per ISA at run time from SSE to AMX and SME.
- OpenVINO CPU Device - States precision defaults per ISA, the int8 path through oneDNN and the streams model that turns cores into throughput.
Threading for inference
- Thread management - Sets the physical-core default, the affinity it implies and the spin-wait controls that trade idle CPU for latency.
- Performance Hints and Thread Scheduling - States the vendor defaults, one thread per core, SMT siblings off, core type by precision and one socket for latency.
- Threadpool: take 2 - Defines the explicit thread pool with CPU masks, strict placement, priority and polling that ggml runs without OpenMP.
- llama-bench - Defines the prompt and generation tests, repetitions and mean with deviation behind any comparable llama.cpp number.
- Dual Epyc Genoa/Turin token generation performance bottlenecks - Traces poor decode scaling across sockets to remote NUMA access from weight placement, with numatop counts as evidence.
When CPU beats GPU
- gpt-j example README (ggml) - Origin of ggml's bandwidth argument, NEON threads saturate a laptop's memory bus, so a GPU on that bus gains nothing.
- llama.cpp Performance Testing - Holds the memory clock sweep with all else fixed, the token rate following it and flattening after a few threads.
- SparAMX: Accelerating Compressed LLMs Token Generation on AMX-powered CPUs - Measures decode kernels on a Xeon as DRAM-bound, so AMX pays at batch one only once bytes per token shrink, with code.
- MLPerf Inference v6.0 Results - Holds CPU-only entries whose logs show the batched Offline case lost to GPU entries at the same accuracy floor.
14. Hardware generations
AMD EPYC
- AMD Next-Generation Zen 4 Core and 4th Gen AMD EPYC Server CPUs - The designers' account of the Zen 4 core and how it yields Genoa, Genoa-X, Bergamo and Siena.
- Testing AMD's Bergamo: Zen 4c Spam - Tests the same-core claim for Zen 4c: cache latency, clock ceiling and core-to-core paths beside clock-matched Zen 4.
- AMD EPYC 9005 Processor Architecture Overview - Defines Turin: Zen 5 or Zen 5c dies, which parts double die-to-IO links, NUMA modes and full-width AVX-512.
- AMD's Turin: 5th Gen EPYC Launched - Measures what wider die-to-IO links and faster DDR5 do for Turin bandwidth, and where latency rose over Genoa.
Arm Neoverse server parts
- AWS Graviton Getting Started - Where AWS states which Neoverse core, ISA revision, mesh, caches and compiler flag each Graviton generation carries.
- NVIDIA Grace Performance Tuning Guide - States the coherency fabric, LPDDR5X fit and MPAM cache and memory partitioning on Grace.
- Arm Neoverse N2 Core Software Optimization Guide - States the timings and fusion rules of the narrower N line core in Cobalt 100 and Yitian.
- Ampere Altra Rev A1 64-Bit Multi-Core Processor Datasheet - Where Ampere states the N1 part: private L2 per core, shared system cache, mesh and DDR4 fit.
Independent measurement across vendors
- Microarchitectural Comparison and In-core Modeling of State-of-the-art CPUs - Measures Neoverse V2, Golden Cove and Zen 4 in-core at fixed clock, and each socket's clock under vector load.
- On the Performance of Cloud-based ARM SVE for Zero-Knowledge Proving Systems - One SVE workload run on Graviton 3, Graviton 4, Yitian and Axion with compiler, runs and spread stated.
- Arm's Neoverse V2, in AWS's Graviton 4 - Measures a sustained rename width below the stated one, cache latencies, mesh behaviour and cross-socket cost on V2.
- ARM's Neoverse N2: Cortex A710 for Servers - Measures structure sizes, cache latencies and mesh behaviour of N2 on Yitian, the core Cobalt 100 carries.
- AmpereOne at Hot Chips 2024: Maximizing Density - Puts vendor slides beside measurements of the predictor, small instruction cache, private L2 and long memory latency.
Intel Xeon
- Technical Overview of the 4th Gen Intel Xeon Scalable Processor Family - Where Intel states what Sapphire Rapids added: larger L2 and L3, DDR5, CXL, AMX and on-die accelerators.
- Sapphire Rapids: Golden Cove Hits Servers - Measures L3 and memory latency across the tiled mesh and the slow clock ramp the vendor overview omits.
- Emerald Rapids: 5th-Generation Intel Xeon Scalable Processors - The designers' statement of what changed: fewer, larger dies, a bigger shared L3, faster DDR5 and socket links.
- A Look into Intel Xeon 6's Memory Subsystem - Measures per-die L3 under sub-NUMA clustering and the die-crossing cost on Granite Rapids beside Turin.
- Benchmarking the Evolution of Performance and Energy Efficiency Across Recent Generations - Bandwidth-bound codes on Sapphire, Emerald and Granite Rapids and Sierra Forest with clocks, SMT and compiler stated.
15. Benchmarks
Methodology and what suites miss
- How Not to Lie with Statistics: The Correct Way to Summarize Benchmark Results - Origin of the rule that normalised results take the geometric mean, which a SPEC ratio and the crimes list rest on.
- Systems Benchmarking Crimes - Checklist of evaluation faults from sub-setting and improper baselines to arithmetic means of ratios, each with a fix.
- Scientific Benchmarking of Parallel Computing Systems - Sets which mean fits costs, rates and ratios, when confidence intervals are owed, and the absolute base a speedup needs.
- Rigorous Benchmarking in Reasonable Time - Decides how many builds, runs and iterations an experiment needs by measuring at which level the variation arises.
- Profiling a warehouse-scale computer - Fleet counter profile showing services stall on instruction fetch and burn cycles in shared routines, which SPEC lacks.
Microbenchmark suites
- Memory Bandwidth and Machine Balance in Current High Performance Computers - Defines sustainable bandwidth as what unit-stride loops get, not bus peak, and machine balance as flops per access.
- STREAM Benchmark Reference Information - Sets the array size rule, timing over repeated trials, and counting bytes a loop asks for, not what the cache moved.
- lmbench: Portable Tools for Performance Analysis - Origin of the one-mechanism-per-test method for memory, system call, pipe and socket latency, and what each leaves out.
- uarch-bench - Isolates memory-level parallelism from load latency as separate tests, with DVFS held off before timing, x86 Linux only.
- nanoBench: A Low-Overhead Tool for Running Microbenchmarks on x86 Systems - Shows why kernel mode with interrupts off matters, removes harness overhead, then recovers cache replacement policies.
Standard suites
- SPEC CPU 2026 Run and Reporting Rules - Defines base against peak, rate against speed, the threading models a speed run may use and an Arm reference machine.
- SPEC CPU: The Next Generation - Where the committee states how workloads were chosen and hardened, and defines the rolling round-robin rate.
- SPEC CPU2026: Characterization, Representativeness, and Cross-Suite Comparison - Measures with counters what each workload stresses on x86 and Arm server parts, beside data-centre and inference suites.
- MLPerf Inference Rules - Fixes model, accuracy floor and query pattern per scenario, so a Server score is throughput under a latency bound.
- DCPerf: An Open-Source, Battle-Tested Performance Benchmark Suite for Datacenter Workloads - Shows standard suites misproject data-centre servers, and states the fleet-matching method the suite is built by.
16. Watchlist
ISA extensions without a shipped server part
- Intel AVX10.2 Architecture Specification - Folds the AVX-512 subsets into one versioned level and adds BF16 and FP8 forms, pending a shipped part and a run.
- Intel Architecture Instruction Set Extensions Programming Reference - Ties APX, with doubled x86 registers, AVX10.2 and AMX FP8 tiles to Diamond Rapids, pending silicon and a public run.
- RISC-V Vector Extension, Version 1.0 - The ratified vector ISA, shipped only as IP and chiplets, pending a socketed server part and a run against Arm or x86.
Kernel paths and generated code
- Extensible Scheduler Class - Lets a BPF program schedule at run time with safe fallback, once a run against the default scheduler states every field.
- io_uring zero copy Rx - Lands payloads straight in user memory on header-splitting NICs, pending the implementer's epoll run naming every field.
- T-MAC - Table-lookup kernels for low-bit weights, with a baseline stated but no frequency, compiler or flags, pending those.
- Faster sorting algorithms discovered using deep reinforcement learning - Generated small sorts shipped in libc++, timed by CPU family with no model, compiler or flags stated, pending those.
Memory and interconnect
- CXL Specification - Defines memory pooled across hosts on a coherent link, with latency so far estimated, pending a run on a shipped pool.
- Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices - Measures expansion devices with frequency and SMT fixed, the nearest run to every field, pending compiler and flags.
- DDR5MDB02 Multiplexed Rank Data Buffer, JESD82-552 - Defines the data buffer behind MRDIMMs, whose vendor bandwidth claims name no method, pending a run against RDIMMs.
Parts without a public measurement
- 6th Gen AMD EPYC Server CPUs - The Zen 6 server family, so far a press release with no shipped part, pending shipment and a public run against Zen 5.
- Intel Xeon 6+ Processors - The E-core-only sockets after Sierra Forest, shipped with vendor multiples footnoted off the page, pending a public run.
- NVIDIA Vera CPU - Custom Arm cores with statically partitioned SMT and no architecture document, pending a specification and a public run.
- Arm Neoverse V3 Core Software Optimization Guide - Vendor timing tables for the core shipped in Graviton 5 and previewed in Cobalt 200, pending a public run on the core.
1. Start here
- Computer Architecture: A Quantitative Approach, 7th Edition - Its pipelining appendix and memory chapters define the hazard, speculation and cache vocabulary the list assumes.
- Optimizing software in C++ - Maps C++ onto pipeline mechanisms and shows why a loop-carried dependency chain, not instruction count, paces a loop.
- Intel Optimization Reference Manual - Its opening chapters show how a shipping x86 core implements the textbook pipeline, each rule tied to a mechanism.
- What Every Programmer Should Know About Memory - Measures the step in cost per access at each cache boundary and the gap a prefetcher hides.
- Memory Barriers: a Hardware View for Software Hackers - Explains why a second core makes loads and stores reorder and what a barrier drains.
- Systems Performance: Enterprise and the Cloud, 2nd Edition - Puts the method before the tools: what to measure, in what order, and how benchmarks mislead.
- Roofline: An Insightful Visual Performance Model for Multicore Architectures - Places a loop from a byte count and a datasheet bandwidth alone, before any counter is read.
- A Top-Down Method for Performance Analysis and Counters Architecture - Defines the split of pipeline slots into front end, bad speculation, back end and retiring, the tree profilers report.
- Performance Analysis and Tuning on Modern CPUs - Walks from a noisy timing to counters to a named bottleneck, applying roofline and top-down to whole programs.
- What Has My Compiler Done for Me Lately? Unbolting the Compiler's Lid - Shows how to read emitted assembly against its source, so each mechanism is checked in a listing, not assumed.
- Performance Ninja
2. One instruction, end to end
Execute
- Focusing Processor Policies via Critical-Path Prediction - Defines the dependence graph of an out-of-order core and the critical path through it that sets run time.
- Instruction tables: latencies, throughputs and micro-operation breakdowns - Measures latency, reciprocal throughput and port assignment per instruction, the weights a dependence graph needs.
- Arm Neoverse V2 Core Software Optimization Guide - States every stage of one Arm core from fetch to issue, then per-instruction latency, throughput and pipe assignment.
- Entropy Decoding in Oodle Data: x86-64 3-Stream Huffman Decoders - Predicts a real decoder loop's cycle count from its carried chain and issue slots, then measures the prediction.
- Faster zlib/DEFLATE decompression on the Apple M1 (and x86) - Predicts a decoder's refill chain from Arm core latencies, fits extra work under it, and measures the gain.
Fetch and decode
- Fetch Directed Instruction Prefetching - The origin of the decoupled front end, where the predictor runs ahead of fetch and drives instruction prefetch.
- Micro-Operation Cache: A Power Aware Frontend for Variable Instruction Length ISA - Introduces the decoded micro-op cache and its power case, the structure both x86 vendors later built.
- Software Optimization Guide for the AMD Zen5 Microarchitecture - Where AMD states when the op cache feeds micro-ops, and the fusion and alignment rules for hot loops.
- The microarchitecture of Intel, AMD, and VIA CPUs - Measures rather than quotes each x86 core's misprediction penalty, micro-op cache and loop buffer behaviour.
- Intel Mitigations for Jump Conditional Code Erratum - States what a microcode fix evicts from the decoded cache and which counters show the fall back to legacy decode.
Memory access and retire
- Memory Dependence Prediction using Store Sets - The origin of memory dependence prediction, which lets a load pass older stores and flushes on a wrong guess.
- Store-to-Load Forwarding and Memory Disambiguation in x86 Processors - Measures which store and load size and offset pairs forward or stall, and which cores predict memory dependences.
- Microarchitecture Optimizations for Exploiting Memory-Level Parallelism - Defines memory-level parallelism as the misses one window overlaps, and shows which core limits cap it.
- Implementing Precise Interrupts in Pipelined Processors - Introduces the reorder buffer and defines a precise exception as in-order commit of out-of-order results.
- Where Do Interrupts Happen? - Measures that an interrupt lands on the oldest unretired instruction, which decides what a sampling profiler blames.
Rename and issue
- An Efficient Algorithm for Exploiting Multiple Arithmetic Units - The origin of tag-based renaming and reservation stations, from which every out-of-order issue queue descends.
- Optimizing subroutines in assembly language - Shows which chains renaming cannot break, from partial registers to flags, and the zero-cost idioms that do.
- Measuring Reorder Buffer Capacity - Sets the user-space method that measures the window and register files, and which idioms take no physical register.
3. Microarchitecture
Branch prediction and speculation
- A Case for (Partially) TAgged GEometric History Length Branch Prediction - Defines the tagged geometric-history predictor shipped designs converge on, and the limit of what it can learn.
- A 64-Kbytes ITTAGE indirect branch predictor - Carries the tagged geometric scheme to indirect jumps and calls, where interpreters and virtual dispatch stall.
- Characterizing the Branch Misprediction Penalty - Defines the penalty as pipeline refill plus window drain, so it exceeds pipeline depth and is not constant.
- Spectre Attacks: Exploiting Speculative Execution - Establishes that predictor state is shared and trainable across contexts, the root of every mitigation and its cost.
- Speculative Execution Side Channel Mitigations - Defines the indirect-branch and store-bypass controls (IBRS, STIBP, IBPB, SSBD) and states which carry a large cost.
Limits of ILP and SMT
- The MIPS R10000 Superscalar Microprocessor - Shows rename, the active list and precise branch recovery fitted together in one shipped out-of-order core.
- Limits of Instruction-Level Parallelism (WRL Research Report 93/6) - Separates perfect from realistic prediction, renaming and aliasing, and shows how little ILP survives the real ones.
- Complexity-Effective Superscalar Processors - Puts circuit delay on wakeup, select and bypass, capping issue width and window size before ILP runs out.
- Simultaneous Multithreading: Maximizing On-Chip Parallelism - The origin of SMT, where another thread fills one thread's idle issue slots at a cost to both.
- A Mechanistic Performance Model for Superscalar Out-of-Order Processors - Turns window size, issue width and miss events into cycles lost, making an ILP limit a cycle count.
Vendor estimates and measured tables
- uops.info - Where each latency, throughput and port entry links to its microbenchmark, so any value can be re-run.
- applecpu: Firestorm Overview - Measured per-instruction tables for an Apple AArch64 core, each entry linked to the counter experiment behind it.
What the manuals leave out
- Performance Speed Limits - Sets the method for finding which hard bound binds a loop, testing its cycles against each in turn.
- uiCA: Accurate Throughput Prediction of Basic Blocks on Recent Intel Microarchitectures - Models the predecoder, decoders, micro-op cache and loop buffer closely enough to predict front-end bound loops.
- Gathering Intel on Intel AVX-512 Transitions - Measures what the vendor leaves untimed in an AVX-512 licence change, a throttled phase then a frequency step.
- Hardware Store Elimination - Finds an undocumented optimisation by its counter signature and bandwidth, and sets the method for what manuals omit.
4. Memory hierarchy
Cache geometry, replacement and misses in flight
- Measuring Cache and TLB Performance and Their Effect on Benchmark Run Times - The origin of the strided-loop method that recovers cache and TLB size, line size, associativity and miss cost.
- Achieving Non-Inclusive Cache Performance with Inclusive Caches - Names inclusion victims as the cost of an inclusive last-level cache, the case for non-inclusive and victim designs.
- Adaptive Insertion Policies for High Performance Caching - The origin of set duelling and bimodal insertion, the adaptive replacement that survives a streaming pass.
- Lockup-Free Instruction Fetch/Prefetch Cache Organization - Origin of the lockup-free cache, whose miss-status registers keep misses in flight, so a stream beats a chase.
Store buffers, ordering and cache-line contention
- x86-TSO: A Rigorous and Usable Programmer's Model for x86 Multiprocessors - The store-buffer model of Intel and AMD ordering, tested on hardware, fixing which reorderings a fence pays for.
- Simplifying ARM Concurrency: Multicopy-Atomic Axiomatic and Operational Models for ARMv8 - The formal model Arm adopted into its architecture, and the record of why the non-multicopy-atomic option was dropped.
- Everything You Always Wanted to Know About Synchronization but Were Afraid to Ask - Measures line-transfer cost between cores by coherence state and distance, the figure behind every contention rule.
- C2C - False Sharing Detection in Linux Perf - The implementers' account of perf c2c, whose worked example reads contended lines, offsets and callers off the report.
Struct layout, software prefetch and page size
- Cache-Conscious Structure Definition - Introduces and measures structure splitting and field reordering on real programs, the origin of hot-cold layout rules.
- CppCon 2014: Data-Oriented Design and C++ - Argues from cache-line utilisation arithmetic that layout must follow the access pattern, the case behind structure of arrays.
- dwarves (pahole) - Prints a struct's holes, padding and cache-line boundaries from DWARF, so a layout is seen rather than guessed.
- When Prefetching Works, When It Doesn't, and Why - Sorts software prefetch into the cases where it helps and hurts, and shows how it mistrains hardware prefetchers.
- Transparent Hugepage Support - The kernel's statement of THP: the enabled and defrag knobs, khugepaged, and the counters showing what was obtained.
- Coordinated and Efficient Huge Page Management with Ingens - Measures the fault latency, memory bloat and unfairness that eager THP promotion causes, and shows what removes them.
TLBs, page walks and prefetchers
- Intel SDM Volume 3A: System Programming Guide, Part 1 - Fixes the paging walk, the page-walk caches, the TLB invalidation rules and what each memory type permits.
- Translation Caching: Skip, Don't Walk (the Page Table) - Defines the page-walk cache design space and shows why cached partial translations let a walk skip page-table levels.
- Improving Direct-Mapped Cache Performance - The origin of the stream buffer that every vendor stream prefetcher descends from, and of the victim cache.
- Arm Neoverse V2 Core Technical Reference Manual - Names an Arm server core's load-side and store-side prefetchers, its TLB levels and the bits that disable them.
5. Measurement
Counters, events and precise sampling
- perf_event_open(2) - Defines the counting and sampling modes every Linux profiler uses, and the sample record fields, branch stack included.
- Instruction-Based Sampling: A New Performance Analysis Technique - Defines skid, why a sample lands after the culprit, and how tagging one op through the pipeline removes it.
- Intel Software Developer Manuals - Defines the architectural counters, what a PEBS sample captures, and why rdtsc counts time, not cycles, under DVFS.
- Processor Programming Reference for AMD Family 1Ah Model 02h - Defines the event encodings and IBS registers for one Zen core, the tables perf's AMD events are derived from.
- perf-arm-spe(1) - Defines Arm SPE as perf drives it, one sampled op in flight, the filters, and what a record holds.
CPU profilers and flame graphs
- Linux perf wiki: Tutorial - The maintainers' walk from perf stat to perf record to perf annotate, with the sample fields each flag sets.
- Intel VTune Profiler Documentation - Home of the user guide and cookbook, where each hardware analysis is defined by the events behind it.