{"id":34504492,"url":"https://github.com/cksfafwefasdf/magicbox","last_synced_at":"2026-05-27T12:31:24.026Z","repository":{"id":188979533,"uuid":"679788316","full_name":"cksfafwefasdf/MagicBox","owner":"cksfafwefasdf","description":"A 32-bit x86 hobbyist OS kernel featuring a Unix-like FS, multi-task scheduling, and a custom Shell.","archived":false,"fork":false,"pushed_at":"2026-05-17T13:36:50.000Z","size":3745,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-05-17T15:50:09.454Z","etag":null,"topics":["c","i386","kernel","operating-system","osdev","unix-like","x86"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cksfafwefasdf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-08-17T16:08:25.000Z","updated_at":"2026-05-17T13:36:53.000Z","dependencies_parsed_at":"2023-08-17T18:55:58.194Z","dependency_job_id":"e42b4e6a-f152-44c7-9660-798a9615d23b","html_url":"https://github.com/cksfafwefasdf/MagicBox","commit_stats":null,"previous_names":["cksfafwefasdf/magicbox"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/cksfafwefasdf/MagicBox","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cksfafwefasdf%2FMagicBox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cksfafwefasdf%2FMagicBox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cksfafwefasdf%2FMagicBox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cksfafwefasdf%2FMagicBox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cksfafwefasdf","download_url":"https://codeload.github.com/cksfafwefasdf/MagicBox/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cksfafwefasdf%2FMagicBox/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33566872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-27T02:00:06.184Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["c","i386","kernel","operating-system","osdev","unix-like","x86"],"created_at":"2025-12-24T02:48:51.793Z","updated_at":"2026-05-27T12:31:24.013Z","avatar_url":"https://github.com/cksfafwefasdf.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MagicBox\n\n**MagicBox** is an early-stage 32-bit x86 operating system prototype. This project serves as a personal practice work to explore the fundamental principles of operating systems, based on classic textbooks and community implementations. It replicates the core pipeline from initial bootloading to a basic user-mode environment.\n\n\u003e **Note**: This project is intended for learning purposes only, utilizing the traditional 32-bit Protected Mode tech stack.\n\n\n\n## 📚 Learning Objectives\n\n- **Bootstrapping \u0026 Environment**: Implemented a minimal MBR and Loader to handle the transition from Real Mode to 32-bit Protected Mode, along with an initial implementation of basic paging.\n- **Minimalist Process Model**: Explored Unix-style process lifecycle management by replicating core system calls such as `fork()` and `execve()`.\n- **Experimental App Deployment**: To simplify the deployment process, a simple **Tar-based** archiving mechanism was implemented. Binaries are written to raw disk sectors and extracted by the kernel into the file system during boot.\n- **Basic Interaction**: Developed a rudimentary command-line Shell that supports internal commands and simple path resolution for external programs (e.g., searching in `/bin`).\n\n\n\n## 🛠 Technical Overview\n\n- **Interrupts \u0026 Synchronization**: Utilizes the **8259A PIC** to handle hardware interrupts. Kernel-level thread synchronization is managed through **mutexes and semaphores**, ensuring safe access to critical sections such as disk I/O operations and process lists.\n\n- **Memory Management: Direct-Map Kernel + VMA-Based User Space**\n\n  This project implements a layered memory-management design that separates kernel-space access paths from user-space virtual memory management:\n\n  - **Physical Layer (Buddy System):** Physical memory is managed by a **Buddy System allocator** (`buddy.c`), replacing the earlier bitmap-only design. This allows scalable page allocation/merge operations and supports large-memory configurations up to the practical 32-bit x86 limit.\n  - **Kernel Low Memory (Direct Mapping):** The kernel no longer allocates ordinary low-memory virtual addresses through a bitmap-managed kernel-vaddr pool. Instead, usable low memory is permanently mapped into the kernel higher-half direct-map window, allowing the kernel to access lowmem pages by simple address translation.\n  - **Kernel High Memory Access (`kmap` / `kunmap`):** Physical pages outside the direct-mapped lowmem range are treated as high memory and are accessed through temporary mappings in a dedicated `kmap` region. This keeps the kernel model simple while still allowing the system to use memory beyond the directly mapped lowmem window.\n  - **User Virtual Space (VMA Framework):** User-space virtual memory is managed through a **VMA (Virtual Memory Area)** framework rather than through per-process virtual bitmaps.\n    - **Demand Paging:** ELF segments, user heap, stack growth, anonymous mappings, and file-backed mappings are all handled lazily. Physical pages are only allocated and mapped when a Page Fault is triggered by actual access.\n    - **Dynamic Management:** Supports VMA insertion, merging, splitting, and gap searching. This provides the infrastructure required for `brk`, `mmap`, `munmap`, lazy file mapping, and Copy-On-Write (COW) during `fork`.\n    - **mmap Layout Policy:** User `mmap` regions are allocated from the high user address range downward (below the reserved stack window), avoiding conflicts with the `brk`-managed heap.\n  - **Kernel Small Allocation:** Kernel small-object allocation still uses an **Arena allocator** built on top of the Buddy System. Arena metadata has been externalized into `struct page`, allowing arena payload pages to remain fully usable.\n  - **User-Space Heap Allocation:** Native user-space `malloc/free` no longer rely on a kernel-side `umalloc` path. Small allocations are managed in user space through `sbrk + arena`, while large allocations are backed by `mmap/munmap`.\n\n- **VFS Implementation \u0026 \"Everything is a File\" Philosophy**\n\n  This project implements a **Virtual File System (VFS)** layer, which attempts to provide a unified interface for hardware access, inter-process communication, and disk storage.\n\n  - **Multi-Filesystem Support**: Uses a function-pointer dispatch design. By implementing `struct super_operations`, `inode_operations`, and `file_operations`, the VFS isolates specific filesystem logic. Currently, it supports both a custom **SIFS** (Simple Index FS) and the **Ext2** specification, allowing different partitions to run different filesystems concurrently.\n  - **Mount Mechanism (Mount/Umount)**: Provides basic mounting functionality. By introducing \"tunneling\" pointers (`i_mount` / `i_mount_at`) within the `struct inode`, the path resolution logic can traverse partition boundaries, supporting nested mount points.\n  - **Dynamic I/O Dispatching**: System calls (such as `read`, `write`, and `ioctl`) bind to specific operation sets during the file-opening phase based on the Inode type, reducing the need for hardcoded type-checking:\n    - **Files \u0026 Directories**: Data block addressing and metadata I/O are handled by the specific Ext2 or SIFS drivers.\n    - **Inter-Process Communication (IPC)**: Anonymous pipes and named pipes (FIFOs) are integrated into the standard file descriptor (FD) management logic.\n    - **Device Management**: Character devices (TTY) and block devices (IDE disks) are mapped as file nodes, accessible via the unified FD interface.\n  - **Metadata Management**: Maintains a global **Inode Hash Table** as a memory cache to ensure the uniqueness of Inode instances. This supports basic features like path backtracking (`getcwd`) and file renaming (`rename`), while ensuring cache consistency during deletion operations like `unlink`.\n\n- **Task Scheduling**: Implements a **Round-Robin** scheduling algorithm. In a design reminiscent of Linux 0.12, a task's `priority` directly determines its allocated clock **ticks**. The system decrements the current task's remaining time slice during timer interrupts, triggering a reschedule only when ticks are exhausted. This non-preemptive approach maintains simplicity while distributing CPU time based on task priority.\n\n- **Signal System**: Provides a basic signal subsystem (supporting `SIGINT`, `SIGKILL`, `SIGCHLD`, `SIGSEGV`, etc.). By manually constructing **user-mode stack frames** within the kernel, the system enables \"upcalls\" to user-defined handlers. Execution context is restored via `sys_sigreturn`, allowing for custom signal handling logic.\n\n- **Resource Recovery**: Implements a two-stage resource reclamation logic. `sys_exit` releases immediate process-private resources (such as FDs), while the destruction of core structures like page directories and kernel stacks is deferred to the parent process during `wait`/`waitpid`. This ensures stable process state synchronization and memory safety.\n\n- **Linux ABI Compatibility**: MagicBox implements a Linux i386 syscall compatibility layer via interrupt `0x80`, specifically targeting the **musl-libc** standard.\n\n  The following environments and tools are currently verified to be functional:\n\n  - **BusyBox:** Supports the **`ash`** shell and a variety of core utilities, including `ls`, `cat`, `grep`, `mkdir`, `mv`, and basic **shell pipes**.\n  - **Tiny C Compiler (TCC):** Capable of running the TCC binary to perform C source code compilation within the MagicBox environment.\n  - **VFS Integration:** Support for various essential filesystem operations including symbolic links, anonymous pipes, and file descriptor control via `fcntl`.\n\n\n\n\n## 📞 System Calls\n\nMagicBox now reserves interrupt `0x80` as a Linux i386 syscall compatibility entry, while the native MagicBox syscall ABI uses interrupt `0x77`.\n\n**Task Management:**\n\n- `fork()` / `clone()`: Creates a new task; `fork()` enforces an isolated address space via Copy-On-Write (COW), while `clone()` offers granular control to either apply COW or share the VM (via `CLONE_VM`).\n- `execve()`: Parses ELF files, builds the initial user stack, and registers VMAs to support on-demand loading.\n- `waitpid()` / `exit()`: Handles process lifecycle synchronization and resource recycling.\n- `setpgid()` / `getpgid()`: Provides basic process group management for shell job control.\n- `alarm()` / `pause()`: Supports simple timed signal delivery and process suspension.\n\n**File System \u0026 IPC:**\n\n- `open()`, `read()`, `write()`, `close()`: POSIX-like file operations.\n- `mount()` / `umount()`: Implements UNIX-like VFS mounting/unmounting, establishing \"tunnels\" between parent directories and child partition roots.\n- `pipe()`: Creates Linux-style anonymous pipes for parent-child communication.\n- `mkfifo()`: Creates persistent named pipe nodes in the file system for unrelated process IPC.\n- `mknod()`: Supports the creation of special files or device nodes (e.g., in `/dev`).\n- `dup2()`: Handles file descriptor redirection, essential for shell pipe implementation.\n- `ioctl()`: A unified device-control interface, currently used for TTY and block-device style control requests.\n\n**Signal Handling:**\n\n- `sigaction()` / `signal()`: Registers signal handlers and configures signal blocking masks.\n- `kill()`: Dispatches signals to specific processes or process groups.\n- `sigprocmask()` / `sigpending()`: Manages the signal blocking state of a process.\n- `sys_sigreturn()`: *(Internal)* Restores execution context after a user-mode signal handler returns.\n\n**Memory Management:**\n\n- `brk()` / `sbrk()`: Provides user heap growth/shrink support through the VMA-based heap region.\n- `mmap()` / `munmap()`: Supports anonymous private mappings and file-backed private mappings, both handled lazily through the page-fault path.\n- Native user-space `malloc()` / `free()`: Small allocations use `sbrk + arena`; large allocations use `mmap`.\n\n**Storage \u0026 Recovery:**\n\n- ~~readraw()~~: *(Deprecated)* Previously used for raw sector access; now superseded by the unified device-file interface, aligning with the \"Everything is a File\" philosophy.\n- `open(\"/dev/...\", ...)`: Replaces legacy raw-disk access. All block devices (e.g., `sda`, `sdb1`) are abstracted as files, enabling user-space tools like `mkfs` and `hexdump` to operate via standard I/O calls.\n  - *Example*: `open(\"/dev/sda\", O_RDONLY)` now provides raw disk access without requiring extra dedicated kernel support.\n\n\n\n## 📂 Project Structure\n\nThe project follows a modular design, separating kernel core logic, hardware drivers, compatibility glue, and user-space applications. Below is an overview of the current directory hierarchy:\n\n```text\n.\n├── boot/               # MBR \u0026 kernel loader (bootstrapping and entry to protected mode)\n├── device/             # Hardware drivers (TTY, keyboard, IDE, IOCTL, timer, console)\n├── fs/                 # VFS layer, file tables, pipes/FIFOs, generic FS logic\n│   ├── ext2/           # Ext2-specific super/inode/file operations\n│   ├── sifs/           # SIFS-specific super/inode/file operations\n│   └── *.c             # VFS and filesystem-independent operations\n├── glue/               # ABI / compatibility glue code (e.g. Linux-style syscall interception)\n├── include/            # Header files organized by scope\n│   ├── arch/           # Architecture-related low-level headers\n│   ├── linux/          # Imported Linux compatibility headers (e.g. i386 syscall numbers)\n│   ├── magicbox/       # Kernel-private subsystem headers\n│   ├── sys/            # Standard C-style headers used inside the project\n│   └── uapi/           # User-kernel ABI definitions (types, ioctls, syscall-facing layouts)\n├── kernel/             # Kernel core (initialization, interrupts, debug, signals)\n├── lib/                # Shared support code\n│   ├── kernel/         # Kernel-mode utility libraries\n│   └── user/           # Native MagicBox user-space wrappers / allocator code\n├── mm/                 # Memory management (buddy allocator, VMA, paging, swap/fault handling)\n├── prog/               # User-space applications and tests\n│   ├── musl_test/      # Linux-ABI / musl-oriented compatibility tests\n│   ├── native_test/    # Tests written against MagicBox's native userspace ABI\n│   └── shell/          # Interactive shell implementation\n├── thread/             # Thread scheduler and synchronization primitives\n├── userprog/           # Process and userspace management (exec, fork, wait/exit, TSS, syscalls)\n├── tool/               # Development helper tools and host-side helper scripts\n├── disk_env/           # Virtual disk images\n├── doc/                # Documentation and preview images\n├── build/              # Build artifacts\n├── makefile            # Master build script\n├── init_disk.sh        # Disk image / partition setup script\n└── install_apps.sh     # Native user-app deployment script\n```\n\n\n\n## 🚀 How to Run\n\n### 1. Prerequisites\n\nEnsure you have the following tools installed on your Linux system:\n\n- **QEMU**: The emulator used to run the OS.\n- **fdisk**: Used by the scripts to partition the virtual hard disk.\n- **GCC \u0026 NASM**: To compile the C and Assembly source code.\n\n### 2. Environment Setup \u0026 Build\n\nFollow these steps in the project root directory:\n\n(1) **Initialize the virtual disk environment** (create images and partitions in the `./disk_env` directory)\n\n```shell\nsh init_disk.sh\n```\n\n(2) By default, the build process does not format the disk. The kernel will automatically initialize a **SIFS** partition upon the first boot. Alternatively, you can pre-format the disk as **Ext2**:\n\n```shell\n# Option A: Standard build (No host-side formatting)\n# The kernel will detect the missing filesystem at boot and \n# automatically initialize a SIFS partition.\nmake all\n\n# Option B: Pre-format the root partition as Ext2\n# This uses the host's mkfs.ext2 tool to prepare the image.\nmake all EXT2=1\n```\n\n*Note: Using `EXT2=1` requires `sudo` privileges on the host for `losetup` and `mkfs.ext2` operations.*\n\n(3) **Install User Applications** Compile and deploy shell utilities into the disk image:\n\n```shell\nsh install_apps.sh\n```\n\n### 3. Launching the OS\n\nNavigate to the `disk_env/` directory. You can choose between a standard run or a high-fidelity hardware simulation.\n\n#### **Standard Run**\n\n```bash\n# argument “-serial none” will disable the uart device \nqemu-system-i386 \\\n  -m 32 \\\n  -drive file=hd60M.img,format=raw,index=0,media=disk \\\n  -drive file=hd80M.img,format=raw,index=1,media=disk \\\n  -serial none\n```\n\nIf you want to use the third disk `sdc`, try:\n\n```shell\nqemu-system-i386 \\\n  -m 32 \\\n  -drive file=hd60M.img,format=raw,index=0,media=disk \\\n  -drive file=hd80M.img,format=raw,index=1,media=disk \\\n  -drive file=hd20M_share.img,format=raw,index=2,media=disk \\\n  -serial none\n```\n\nIf you want to use the UART, try:\n\n```shell\nqemu-system-i386 \\\n  -m 32 \\\n  -drive file=hd60M.img,format=raw,index=0,media=disk \\\n  -drive file=hd80M.img,format=raw,index=1,media=disk \\\n  -nographic -serial mon:stdio\n```\n\n#### **Hardware Simulation Mode**\n\nTo better simulate real hardware behavior (including sync disk I/O and precise clocking), use the following:\n\n```bash\nqemu-system-i386 \\\n  -m 32 \\\n  -drive file=hd60M.img,format=raw,index=0,media=disk,cache=directsync \\\n  -drive file=hd80M.img,format=raw,index=1,media=disk,cache=directsync \\\n  -rtc base=localtime,clock=vm \\\n  -icount shift=auto,sleep=on \\\n  -serial none \\\n  -boot c\n```\n\n\u003e **Note**:\n\u003e\n\u003e - `hd60M.img`: Contains the MBR, Loader, and the Kernel.\n\u003e - `hd80M.img` / `hd20M_share.img` : Secondary data disks. To use the partitions within these disks, they must be initialized (via `mkfs.sifs` or `mkfs.ext2`) and then attached to the VFS tree using the `mount` command.\n\n\n\n**Memory Support:** Thanks to the newly implemented **Buddy System**, the `-m` parameter now supports up to **2048** (2GB).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcksfafwefasdf%2Fmagicbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcksfafwefasdf%2Fmagicbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcksfafwefasdf%2Fmagicbox/lists"}