{"id":51716593,"url":"https://github.com/appleweiping/stanford-cs110-syscore","last_synced_at":"2026-07-17T03:36:33.085Z","repository":{"id":370362993,"uuid":"1293102196","full_name":"appleweiping/stanford-cs110-syscore","owner":"appleweiping","description":"From-scratch C++ implementations of Stanford CS110 (Principles of Computer Systems): Unix v6 filesystem, multiprocessing, stsh shell w/ job control, ThreadPool, RSS aggregator, HTTP proxy+cache, MapReduce — all verified in WSL2.","archived":false,"fork":false,"pushed_at":"2026-07-09T00:11:24.000Z","size":84,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-09T02:07:37.280Z","etag":null,"topics":["concurrency","cpp","csdiy","networking","operating-systems","stanford-cs110","systems-programming"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/appleweiping.png","metadata":{"files":{"readme":"README.md","changelog":"news-aggregator/Makefile","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-08T05:34:09.000Z","updated_at":"2026-07-09T00:12:08.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/appleweiping/stanford-cs110-syscore","commit_stats":null,"previous_names":["appleweiping/stanford-cs110-syscore"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/appleweiping/stanford-cs110-syscore","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/appleweiping%2Fstanford-cs110-syscore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/appleweiping%2Fstanford-cs110-syscore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/appleweiping%2Fstanford-cs110-syscore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/appleweiping%2Fstanford-cs110-syscore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/appleweiping","download_url":"https://codeload.github.com/appleweiping/stanford-cs110-syscore/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/appleweiping%2Fstanford-cs110-syscore/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35566571,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-17T02:00:06.162Z","response_time":116,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrency","cpp","csdiy","networking","operating-systems","stanford-cs110","systems-programming"],"created_at":"2026-07-17T03:36:32.459Z","updated_at":"2026-07-17T03:36:33.076Z","avatar_url":"https://github.com/appleweiping.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Stanford CS110 — Principles of Computer Systems\n\n\u003e From-scratch C/C++ implementations of the CS110 systems-programming\n\u003e assignments — filesystems, multiprocessing, a shell with job control,\n\u003e concurrency, and networking — an independent, from-scratch build of\n\u003e **CS110 — Principles of Computer Systems** (Stanford), part of a\n\u003e [csdiy.wiki](https://csdiy.wiki/) full-catalog build.\n\n![status](https://img.shields.io/badge/status-complete-brightgreen)\n![language](https://img.shields.io/badge/C++17-informational)\n![license](https://img.shields.io/badge/license-MIT-blue)\n\n## Overview\n\nCS110 teaches how real systems are built out of the Unix system-call layer:\nfilesystems, processes and pipes, signals and job control, threads and\nsynchronization, and networking. This repo implements **all 8 assignments** in\nidiomatic C++17 on raw POSIX APIs — no course helper libraries; where the course\nships a private library (threadpool, socket++, HTTP classes) or a proprietary\ndataset, a minimal working equivalent is implemented from scratch and a builder\ngenerates real spec-conformant data. Everything compiles `-Wall -Wextra -Werror`\nclean and **runs end-to-end** in WSL2, each with an automated test that captures\nreal output into [`results/`](results/).\n\n## Results (measured on WSL2, Ubuntu 24.04, g++ 13.3, 16-core CPU, libxml2 2.9.14)\n\n| # | Assignment | What it does | Result (measured) |\n|---|---|---|---|\n| 1 | **Amazon Reviews Search** | mmap'd binary DB + keyword index, pointer arithmetic, STL binary search | **10/10** tests, **valgrind-clean**; phrase `\"works great\"`→3 vs AND `works great`→4 |\n| 2 | **Unix v6 Filesystem** | layered read-only V6 fs: inode → file → directory → pathname | **20/20** assertions; `ILARG` indirect addressing verified on a 20-block file |\n| 3 | **Multiprocessing** | `subprocess`, `pipeline`, self-throttling `farm` | subprocess PASS, `sort\\|uniq -c` correct, farm across **16 CPUs** (`1048576 = 2^20`) |\n| 4 | **stsh (Stanford Shell)** | pipelines, redirection, job control, signals | **9/9** functional + **6/6** pty job-control (Ctrl-Z/Ctrl-C forwarding) |\n| 5 | **RSS News Aggregator** | multithreaded fetch + libxml2 parse + inverted index | **7/7**; de-dup proven by access log (shared article fetched **once**) |\n| 6 | **ThreadPool** | dispatcher + workers, hand-rolled semaphore | **5/5**; 16×100 ms tasks on 8 workers in **202 ms** (vs 1600 ms serial); **TSan-clean** |\n| 7 | **HTTP Web Proxy + Cache** | concurrent forward proxy, cache, blocklist | **9/9**; cache causal (origin hit **once**), **25-way** concurrency, blocklist → 403 |\n| 8 | **MapReduce** *(optional)* | single-node map → hash-shuffle → reduce | **7/7**; counts exact, fixed a real concurrent-`fork` fd leak (5/5 stable) |\n\n## Implemented assignments\n\n- [x] **assign1 — Amazon Reviews Search** — keyword search over mmap'd binary review DB + sorted keyword index (`lower_bound`, phrases, `set_intersection`)\n- [x] **assign2 — Reading Unix v6 Filesystems** — bottom-up V6 reader with a self-contained `mkfs_v6` image builder\n- [x] **assign3 — All Things Multiprocessing** — `subprocess`/`pipeline`/`farm` on fork/exec/pipe + `SIGSTOP`/`SIGCONT` throttling\n- [x] **assign4 — stsh, the Stanford Shell** — arbitrary pipelines, `\u003c`/`\u003e`, `\u0026`, `jobs`/`fg`/`bg`/`slay`/`halt`/`cont`, terminal + signal job control\n- [x] **assign5 — RSS News Feed Aggregator** — two-stage ThreadPool crawler, libxml2 RSS/HTML parsing, inverted index, per-server throttle, de-dup\n- [x] **assign6 — ThreadPool** — reusable dispatcher+worker pool, hand-rolled counting semaphore, ThreadSanitizer-clean\n- [x] **assign7 — HTTP Web Proxy and Cache** — concurrent forward proxy on raw sockets, in-memory cache (Cache-Control), regex blocklist, `x-forwarded-for`\n- [x] **assign8 — MapReduce** *(optional/ungraded)* — real map/shuffle/reduce mechanics with exec'd mapper/reducer; cluster layer replaced by local processes\n\n## Project structure\n\n```\nstanford-cs110-syscore/\n├── amazon-reviews/      # assign1: mmap binary DB + keyword search + mkamazondb builder\n├── unix-v6-filesystem/  # assign2: V6 fs reader (inode/file/directory/pathname) + mkfs_v6\n├── multiprocessing/     # assign3: subprocess, pipeline, farm (+ factor.py worker)\n├── stsh/                # assign4: shell; tests/ has functional + pty job-control tests\n├── news-aggregator/     # assign5: RSS aggregator (fetch, xml-utils, aggregate)\n├── thread-pool/         # assign6: ThreadPool + semaphore + tptest\n├── http-proxy/          # assign7: proxy, http model, cache, blocklist + local-origin tests\n├── mapreduce/           # assign8: mr engine + mapper/reducer\n├── results/             # captured real run output for every assignment\n├── LICENSE              # MIT (covers this repo's own code)\n└── README.md\n```\n\n## How to run\n\nEverything is Linux-only (POSIX). Build and run inside **WSL2 Ubuntu**:\n\n```bash\nsudo apt-get install -y build-essential libxml2-dev valgrind   # libxml2 only needed by assign5\ncd stanford-cs110-syscore\n\n# each assignment builds and self-tests independently:\nfor d in amazon-reviews unix-v6-filesystem multiprocessing stsh \\\n         news-aggregator thread-pool http-proxy mapreduce; do\n  ( cd \"$d\" \u0026\u0026 make test )\ndone\n```\n\nEach directory has its own `README.md` with design notes and its exact test\ncommand.\n\n## Verification\n\nEvery assignment ships an automated test whose real output is captured under\n[`results/`](results/):\n\n- `results/assign1_amazon.txt` … `results/assign8_mapreduce.txt`.\n- Tests are self-checking (assert exact values) and cover the substantive\n  behavior: V6 indirect-block addressing, farm load-balancing across all CPUs,\n  pty-driven Ctrl-Z/Ctrl-C job control, causal proof of proxy caching and\n  aggregator de-duplication via origin access logs, ThreadSanitizer- and\n  valgrind-clean runs.\n\n## Tech stack\n\nC++17 · POSIX system calls (`fork`/`execvp`/`pipe`/`dup2`/`mmap`/`setpgid`/\n`tcsetpgrp`/`waitpid`/`sigsuspend`, BSD sockets) · `std::thread`/`mutex`/\n`condition_variable` · libxml2 (RSS/HTML) · GNU Make · GCC 13.3 · WSL2 Ubuntu ·\ntested with ThreadSanitizer and Valgrind.\n\n## Key ideas / what I learned\n\n- **Descriptor discipline** is everything in multiprocessing: a single leaked\n  pipe write-end wedges EOF detection (assign3, and the `O_CLOEXEC` fix in\n  assign8's concurrent forks).\n- **Job control** is process groups + `tcsetpgrp` + forwarding terminal signals\n  to the foreground group, with `sigsuspend` to wait race-free (assign4).\n- **Concurrency correctness**: coordinating a pool with counting semaphores and\n  proving it race-free under TSan (assign6), and throttling per-server\n  connections while sharing an index safely (assign5).\n- **Binary formats \u0026 mmap**: reading offset-indexed records with pure pointer\n  arithmetic, and binary-searching a sorted keyword table (assign1, assign2).\n- **Networking**: a forward proxy is HTTP parsing + socket plumbing + a cache\n  keyed on the request, made causal-testable against a local origin (assign7).\n\n## Credits \u0026 license\n\nBased on the assignments of **CS110 — Principles of Computer Systems** by Chris\nGregg, Jerry Cain, and Nick Troccoli (Stanford). This repository is an\nindependent educational reimplementation; all course materials and\nspecifications belong to their original authors. Original code in this repo is\nreleased under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fappleweiping%2Fstanford-cs110-syscore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fappleweiping%2Fstanford-cs110-syscore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fappleweiping%2Fstanford-cs110-syscore/lists"}