{"id":22777790,"url":"https://github.com/a1exwang/modern_05","last_synced_at":"2026-01-24T01:32:05.343Z","repository":{"id":95285230,"uuid":"362848825","full_name":"a1exwang/modern_05","owner":"a1exwang","description":"A modern toy OS using up-to-date technology (AMD64, UEFI, Multi-Core CPU, PCI Express, ACPI/SATA, ZFS, Modern C++)","archived":false,"fork":false,"pushed_at":"2021-10-16T18:40:51.000Z","size":502,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-16T14:23:27.869Z","etag":null,"topics":["cpp","hobby-os","kernel"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/a1exwang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-29T14:40:11.000Z","updated_at":"2023-12-20T16:21:39.000Z","dependencies_parsed_at":"2023-04-21T11:48:53.597Z","dependency_job_id":null,"html_url":"https://github.com/a1exwang/modern_05","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a1exwang%2Fmodern_05","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a1exwang%2Fmodern_05/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a1exwang%2Fmodern_05/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a1exwang%2Fmodern_05/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/a1exwang","download_url":"https://codeload.github.com/a1exwang/modern_05/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249554619,"owners_count":21290462,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","hobby-os","kernel"],"created_at":"2024-12-11T19:17:21.473Z","updated_at":"2026-01-24T01:32:05.314Z","avatar_url":"https://github.com/a1exwang.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# unnamed OS\n## A modern OS prototype using up-to-date technology (AMD64, UEFI, Multi-Core CPU, PCI Express, ACPI/SATA, ZFS, Modern C++)\n\n## Principles\n- Only support up-to-date technology. Don't even consider backward compatibility.\n- Roughly explore different concepts of an OS rather than doing one and do it best.\n- Solve the most valuable problem first and add TODOs or asserts for the rest.\n- Keep the code short and straight-forward at the cost of robustness (for the error cases, just let it kernel panic if the user can prevent them).\n\n## Prerequisites\n- AMD64 CPU (this OS only supports AMD64 architecture, no x86 support)\n- GCC 10 (any version should be fine as long as you can compile my code, currently no support for GCC 11 because FUTEX issue)\n- An environment for the OS to run, either of the three\n - qemu 6.0 + TianoCore EDK 2 (For EFI support in qemu, this OS only support EFI, no BIOS support)\n - VirtualBox 6.x (VirtualBox has native EFI support, but VirtualBox does not support RTL8139, so you do not have a working NIC driver, probably)\n - physical machine (AMD64 CPU, a motherboard with Q35 chipset, RTL8139 NIC connected via PCI, AHCI SATA controller and a SATA HDD)\n- Some common Linux utilities (CMake, Make, udisksctl)\n\n## Build\n\n### QEMU\n\n``` \nmkdir cmake-build-debug \u0026\u0026 cd cmake-build-debug\ncmake ..\ncd -\n./qemu\n```\n\n### Launch QEMU and use gdb to debug the kernel\n\n```\n$ ./qemu.sh -S -s \n$ gdb\n\u003e\u003e\u003e target remote :1234\n\u003e\u003e\u003e file kernel\n\u003e\u003e\u003e c\n```\n\n### VirtualBox\n\n``` \n./vbox.sh\n```\n\n## TODOs\n- boot\n - ~~EFI~~\n - initramfs, embed in kernel image, depends on: VFS\n\n- memory\n - ~~buddy page allocator~~\n - kernel dynamic memory allocator based on buddy allocator (maybe slab)\n - ~~virtual memory(setup gdt,page table)~~\n - uncached \u0026 low memory pool, depends on page allocator/slab allocator\n - IOMMU\n\n- irq\n - ~~close irq on interrupt handler~~\n - ~~clock interrupt~~\n - register all exception handlers\n - ~~pretty exception dumper, depends on C++ Exception/libunwind support~~\n - ~~irq register/unregister, general irq device polling~~\n - refactor IOAPIC\n\n- process\n - ~~kthread~~\n - ~~user space process~~\n - ~~basic syscall~~\n - smp\n - **fork**\n - **exec**\n - **load simple ELF**, depends on VFS and a file system\n - scheduler, currently we have a simple round-robin scheduler\n - ~~spinlock~~\n - **sleep**\n\n- fs\n - **+vfs**\n - mkdir, readdir, unlink\n - open, read, write\n - dcache, page cache\n - ~~memfs, depends on vfs~~\n - ~~simple in-memory file~~\n - GPT partition table, depends on blockdev\n - ext2, depends on GPT and blockdev\n - stdio\n - zfs, too complicated (maybe in later versions)\n\n- kernel library\n - **kernel string library, sprintf**\n - libunwind port/ ~~write a libunwind-like library~~\n\n- userspace library\n - userspace syscall wrapper\n - port mlibc\n - ~~mlibc can compile with empty syscall wrapper~~\n - mlibc can run hello world with some syscall wrapper\n - mlibc can run busybox\n - port busybox running on mlibc\n - ~~compile ok~~\n \n- bus driver\n - ~~PCI~~\n - MSI/MSI-X (no such device to test)\n - ~~PCI express~~\n - **USB**\n\n- network\n - ~~RTL8139 NIC driver~~\n - support 8139C+ to zero-copy\n - **general L2 driver**\n - ~~ARP~~\n - sometimes arp thread stuck\n - IP, IPv6\n - UDP\n - TCP (maybe we won't implement a full-fledged TCP, implement just enough for SSH and HTTP)\n - BSD sockets\n\n- block device\n - ~~AHCI read sector~~\n - AHCI write sector\n - AHCI DMA mode\n - general block device/in memory block dev\n\n- devices\n - VGA\n - keyboard\n - ~~serial port~~\n\n- TTY\n - port my terminal emulator to implement TTY\n\n- infrastructure\n - C++ exception, depends on libunwind\n - ~~static initialization \\(maybe we need __attribute to put different var in different section, to init them in different stages), may depend on elf?~~\n - ~~malloc/free/new/free, depends on slab allocator~~\n - debug symbol, depends on libunwind, elf and dwarf\n - ~~stack dumper, using my simple unwind algo~~\n - full fledged unwinder\n\n- misc\n - use gdb to debug qemu with gdbserver to debug kernel 😒\n - fix compiler warnings\n - compiler attributes: iomem, address_space(x), etc.\n - unit test\n - small bugs\n - busybox/mlibc not built everytime\n\n- docs\n - add a working example, maybe screencast\n - write a script to set up disk image\n - can we run this in a browser (seems not practical since there seems to be no complete AMD64 emulator in browser, no device emulation, no IOMMU, etc)\n \n- userspace porting\n\n## Book/Document List\nThis are the book and documents I read when writing this OS. Some just thoroughly read through.\n\n#### CPU\n- AMD64 Architecture Programmer's Manual Volume 1-3\n - This manual is the most useful when writing initial platform dependent components like memory management, IRQ, IO port, assembly, etc.\n\n#### IO, IRQ\n- Advanced Configuration and Power Interface Specification Version 6.2 (ACPI)\n - This is useful when decoding various ACPI tables from EFI to get hardware information.\n- ACPI Component Architecture User Guide and Programmer Reference Revision 6.2 by Intel (ACPICA)\n - This library will help you interpret the OS independent byte code of ACPI\n- AMD I/O Virtualization Technology (IOMMU) Specification Rev 3.06\n - Useful when you need to virtualize your 32-bit DMA addresses to use to full 64 bit address for legacy devices that do not support 64-bit BAR address or avoid bounce buffers.\n- Intel® I/O Controller Hub 9 (ICH9) Family Datasheet\n - ICH9 chipset. We support this because this is the chipset qemu emulates when using EFI\n- Intel 82093AA I/O ADVANCED PROGRAMMABLE INTERRUPT CONTROLLER (IOAPIC) MANUAL\n - IOAPIC to route your ISA IRQs to local APIC, so it will be finally handled by your ISR.\n\n#### Device\n- PCI EXPRESS BASE SPECIFICATION, REV. 3.0\n - PCI Express spec, backward compatible with PCI spec. We need a PCI bus to connect all the devices and buses anyway.\n\n#### Network\n- RTL8139D Datasheet by Realtek\n - This is the simplest NIC supported by qemu. Just simplicy. It is 100M full-duplex NIC.\n- RFC 826: An Ethernet Address Resolution Protocol\n - ARP protocol to map between Ethernet address and IP address\n\n#### Block device and file system\n- Serial ATA Advanced Host Controller Interface (AHCI) 1.3.1\n - SATA spec. We need SATA HDDs because why not. We are running on a PC, why not HDDs?\n- ZFS On-Disk Specification by Sun Microsystems\n - ZFS sounds awesome. We don't have the license issue as Linux has. Why not use ZFS as we need a file system anyway.\n\n#### Misc\n- Unified Extensible Firmware Interface Specification 2.6\n - UEFI spec. BIOS is dead, long live the BIOS. Very useful when you write the bootloader and some of the device enumeration. Way easier to use than BIOS.\n- Intel® 64 Architecture x2APIC Specification\n - We will need it one day, to support SMP?\n- Executable and Linking Format (ELF) Specification Version 1.2\n - ELF spec, not much to say. I use ELF as my executable format.\n- Linux Kernel Development 3rd Edition, a.k.a LKD\n - Good book to reference when you copy Linux's implementations\n\n#### When things do work out...\n1. Linux source code is your best teacher. For x86 specific details, see Linux 1.0 because it is simpler. For device drivers, just read the mainline kernel.\n1. QEMU is another teacher because you can see the implementation of the actual hardware (though not recommended to depend on implementation detail, it is a good start for your bugs and questions)\n1. OSDev is your friend. If you cannot find the info from your teachers efficiently (maybe Linux or QEMU is too big), go to them. They may not answer your questions completely. But their articles are easier to get your hands on.\n1. GDB, objdump and hexdump is your final hope.\n1. If hope is lost, go to sleep and then to the first step.\n\n## Sample output for QEMU\n- have a usable libc\n - implement compatible syscall and porting glibc\n - port a libc with syscall stubs\n - implement my own libc\n```text\nShell\u003e fs0: \nFS0:\\\u003e boot_loader.efi \nTextOut-\u003eOutputString() \nHello, world! \ncs: 0038 \nAllocated 0x2100 pages starting from 0x79E36000 \nDescriptor version/size/count: 0x1/0x30/0x7B \nConventional memory regions: \n Type Pad PhysicalStart VirtualStart NPages Attr \n 0x03 0x00000000 0x0000000000000000 0x0000000000000000 0x00000001 0x0000000F \n 0x07 0x00000000 0x0000000000001000 0x0000000000000000 0x0000009F 0x0000000F \n 0x07 0x00000000 0x0000000000100000 0x0000000000000000 0x00000700 0x0000000F \n 0x0A 0x00000000 0x0000000000800000 0x0000000000000000 0x00000008 0x0000000F \n 0x07 0x00000000 0x0000000000808000 0x0000000000000000 0x00000008 0x0000000F \n 0x0A 0x00000000 0x0000000000810000 0x0000000000000000 0x000000F0 0x0000000F\n...\n\nKernel entry: 55 48 89 E5 48 83 EC 10 48 89 7D F8 E8 E2 0C 01 \nkernel loaded at virt: 0xFFFF800000100000, phy: 0x7A036000 \nFound ACPI 2.0 RSDP at 7FB7E014 \nACPI OEM string 'BOCHS ', version 2, rsdt = 0x7FB7D074, xsdt = 0x7FB7D0E8 \nhello serial from bootloader \nbefore setting cr3 to 7d36c000 \npml4e 0 7fc02023 \n pdpe 0 7fc03023 \n pdpe 1 7fc04023 \n pdpe 2 7fc05023\n...\n\nkernel page table setup done \nHello from kernel! \ncs = 0x38 \nds = 0x30 \nes = 0x30 \nfs = 0x30 \ngs = 0x30 \nss = 0x30 \nrax = 0xffff800000505f30 \nrcx = 0x3f8 \nrdx = 0x1 \nrbx = 0x7ff17488 \nrsi = 0xa \nrdi = 0x3f8 \nrbp = 0xffff800000505ed0 \nrsp = 0xffff800000505e00 \nrflags = 0x46 \ncr0 = 0x80010033 \ncr3 = 0x7d36c000 \ncr4 = 0x668 \nGDT address = 0x7f9ee698 \nSegment descriptor 0x38 dpl 0 long_mode 1 default_operand_size 0 \nIDT address = 0x7f274018 limit 0xfff \ncr3 7d36c000 \npml4e 0 7fc02023 \npdpe 0 7fc03023 \npml4e 100 7d36d023 \ntotal pages: 0x40000 \nsetting cs = 0x10 \nsetting all data selectors to 0x18 \nsetting up user code/data selectors \nloading tr 0x30 \nmoving page table to kernel, new cr3 = 0x7a455000 \nMemory segments: \nAvailable memory sections = a, size = 5159196KiB \nmax_pages_addr: 4000000, max_pages 40000 \nBuddy allocator at 0xffff800000d29040 \n bucket 16 size = 64KiB: \n bucket 17 size = 128KiB: \n bucket 18 size = 256KiB: \n bucket 19 size = 512KiB: \n bucket 20 size = 1MiB: \n bucket 21 size = 2MiB: \n bucket 22 size = 4MiB: \n...\nLocal APIC ID = 0 base = 0xffff8000fee00000 \nACPI extended system description table size = 84, entries = 6 \n ACPI table 0 FACP \n ACPI table 1 APIC \n ACPI table 2 HPET \n ACPI table 3 MCFG \n...\ndisabling 8259 PIC \nIO APIC 0x0 0xfec00000 0x0 \nIOAPIC0 version 0x20 0x18 \n IOAPIC 0, 2040 0 \n IOAPIC 1, 2041 0 \n IOAPIC 2, 2042 0 \n...\nEnumerating PCI devices: \nSegment group b0000000 00 ff \n bus 0x0 slot 0x0 \n function 0x0 vendor 0x8086 device 0x29c0 header 0x0 class 0x6 subclass 0x0 IRQ line 0xff IRQ pin 0x0 \n bus 0x0 slot 0x1 \n function 0x0 vendor 0x1234 device 0x1111 header 0x0 class 0x3 subclass 0x0 IRQ line 0xff IRQ pin 0x0 \n bus 0x0 slot 0x2 \n function 0x0 vendor 0x10ec device 0x8139 header 0x0 class 0x2 subclass 0x0 IRQ line 0xb IRQ pin 0x1 \n BAR[0] = 0x6000, size = 0x100 IO space \n BAR[1] = 0xc1041000, size = 0x100 Memory space \n...\nuser image 8000000 stack 0x8010000 \nstarting process pid = 1 '' \nhas tmp_start, going for it \nmain process started \nmain process creating process 2 \nuser image 8020000 stack 0x80b0000 \nmain process loading \ninit elf size = 1900, start bytes 0x7f 0x45 \nelf has 5 sections at 0x40 \n p_offset p_filesz p_vaddr p_memsz \n 0x1000 0xc4 0x110000 0xc4 \nentrypoint at 110000 \nELF file loaded \nwrite() from pid 1 content 'hello from uSeR5p@ze' \nstarting process pid = 2 '2!' \nhas tmp_start, going for it \nthread2 started \nRTL8139 rx 0x0 0xd9 capr 0xfff0 cbr e0 \n 0000000000000000 | 01 00 5e 7f ff fa ea a3 3e 38 ca 4d 08 00 45 00 | ..^.....\u003e8.M..E. \n 0000000000000010 | 00 c7 14 73 40 00 01 11 c8 a3 ac 14 00 01 ef ff | ...s@........... \n 0000000000000020 | ff fa ec 8b 07 6c 00 b3 ab 9f 4d 2d 53 45 41 52 | .....l....M-SEAR \n 0000000000000030 | 43 48 20 2a 20 48 54 54 50 2f 31 2e 31 0d 0a 48 | CH * HTTP/1.1..H \n 0000000000000040 | 4f 53 54 3a 20 32 33 39 2e 32 35 35 2e 32 35 35 | OST: 239.255.255 \n 0000000000000050 | 2e 32 35 30 3a 31 39 30 30 0d 0a 4d 41 4e 3a 20 | .250:1900..MAN: \n 0000000000000060 | 22 73 73 64 70 3a 64 69 73 63 6f 76 65 72 22 0d | \"ssdp:discover\". \n 0000000000000070 | 0a 4d 58 3a 20 31 0d 0a 53 54 3a 20 75 72 6e 3a | .MX: 1..ST: urn: \n 0000000000000080 | 64 69 61 6c 2d 6d 75 6c 74 69 73 63 72 65 65 6e | dial-multiscreen \n 0000000000000090 | 2d 6f 72 67 3a 73 65 72 76 69 63 65 3a 64 69 61 | -org:service:dia \n 00000000000000a0 | 6c 3a 31 0d 0a 55 53 45 52 2d 41 47 45 4e 54 3a | l:1..USER-AGENT: \n 00000000000000b0 | 20 47 6f 6f 67 6c 65 20 43 68 72 6f 6d 65 2f 39 | Google Chrome/9 \n 00000000000000c0 | 30 2e 30 2e 34 34 33 30 2e 39 33 20 4c 69 6e 75 | 0.0.4430.93 Linu \n 00000000000000d0 | 78 0d 0a 0d 0a f0 26 d5 9f | x.....\u0026......... \n \nRTL8139 rx 0xe0 0x40 capr 0xd0 cbr 124 \n 0000000000000000 | 0a 01 0e 0a 01 0e ea a3 3e 38 ca 4d 08 06 00 01 | ........\u003e8.M.... \n 0000000000000010 | 08 00 06 04 00 02 ea a3 3e 38 ca 4d ac 14 00 01 | ........\u003e8.M.... \n 0000000000000020 | 0a 01 0e 0a 01 0e ac 14 00 03 00 00 00 00 00 00 | ................ \n 0000000000000030 | 00 00 00 00 00 00 00 00 00 00 00 00 59 f2 c3 42 | ............Y..B \n \nRTL8139 tx ok \n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa1exwang%2Fmodern_05","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fa1exwang%2Fmodern_05","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa1exwang%2Fmodern_05/lists"}