{"id":51164294,"url":"https://github.com/dorsaroh/kernels","last_synced_at":"2026-06-26T17:02:24.839Z","repository":{"id":293899328,"uuid":"985024312","full_name":"DorsaRoh/Kernels","owner":"DorsaRoh","description":"Kernels from scratch","archived":false,"fork":false,"pushed_at":"2025-05-27T17:48:52.000Z","size":486,"stargazers_count":9,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-04T22:03:24.608Z","etag":null,"topics":["kernel","kernel-development","kernels"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DorsaRoh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-16T23:33:34.000Z","updated_at":"2025-06-14T07:05:37.000Z","dependencies_parsed_at":"2025-05-17T20:40:08.284Z","dependency_job_id":null,"html_url":"https://github.com/DorsaRoh/Kernels","commit_stats":null,"previous_names":["dorsaroh/kernels"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/DorsaRoh/Kernels","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DorsaRoh%2FKernels","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DorsaRoh%2FKernels/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DorsaRoh%2FKernels/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DorsaRoh%2FKernels/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DorsaRoh","download_url":"https://codeload.github.com/DorsaRoh/Kernels/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DorsaRoh%2FKernels/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34825611,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-26T02:00:06.560Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kernel","kernel-development","kernels"],"created_at":"2026-06-26T17:02:22.021Z","updated_at":"2026-06-26T17:02:24.831Z","avatar_url":"https://github.com/DorsaRoh.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kernels from Scratch\n\nImplements \u0026 explains a kernel, **assuming you know absolutely nothing.** \n\nEnjoy!\n1. [What is a Kernel?](#1-kernel)\n2. [Fundamentals \u0026 Memory Management](#1-fundamentals--memory-management)\n\n\n\n### 1. Kernel\nThe kernel is the core part of your operating system (OS) that sits between your computer's hardware and all other software (like apps or the user interface).\n\n- It abstracts hardware differences, giving software a consistent, standardized environment to run on.\n\nThis means developers don’t need to write different code for each type of hardware.\n\nAnalogy:\nThink of the kernel like your house’s plumbing system.\nIt doesn’t matter where the water comes out—your shower, kitchen sink, or garden hose—the plumbing makes sure it flows properly, no matter the source.\nSimilarly, the kernel ensures that software can access and use hardware resources smoothly and reliably.\n\n\n## 2. Fundamentals \u0026 Memory Management\n\n### Pointers and references (C++):\n\n#### **Reference**: \nan alias for an existing object. Is essentially identical to the object being referenced.\n\n   - It does **not** store its own value or memory address. It directly refers to another variable's address in memory.\n\n   Think of it like a nickname for the same person—you’re referring to the same underlying thing.\n\n**Any change made through the reference affects the original variable and vice versa.**\n\n\n```\n// regular types\nint             // a normal int type (not a reference)\nint\u0026            // a reference to an int object\ndouble\u0026         // a reference to a double object\nconst int\u0026      // a reference to a const int object\n```\n\u003cbr\u003e\n\nThe syntax for references `type\u0026 ref` is not to be confused with the address-of operator `\u0026x`, which returns the memory address of variable x.\n\nFor example,\n\n```\n#include \u003ciostream\u003e\n\nint main()\n{\n    int x{ 5 };\n    std::cout \u003c\u003c x \u003c\u003c '\\n';  // print the value of variable x\n    std::cout \u003c\u003c \u0026x \u003c\u003c '\\n'; // print the memory address of variable x\n\n    return 0;\n}\n```\nOne run of this prints:\n```\n5\n0027FEA0\n```\n\n#### Tip:\nThe \u0026 symbol tends to cause confusion because it has different meanings depending on context:\n\n- When following a type name, \u0026 denotes an lvalue reference: `int\u0026 ref`.\n- When used in a unary context in an expression, \u0026 is the address-of operator: `std::cout \u003c\u003c \u0026x`.\n- When used in a binary context in an expression, \u0026 is the Bitwise AND operator: `std::cout \u003c\u003c x \u0026 y`.\n\n\n\u003cbr\u003e\n\n[See code here](./Fundamentals/C++/reference.cpp)\n\n\n---\n\n#### Dereference operator (*)\n\nReturns the value at a given memory address.\n\nFor example,\n\n```\nint main(){\n   int x = 5;\n   std::cout \u003c\u003c x;      // print the value\n   std::cout \u003c\u003c \u0026x;     // print the memory address\n\n   std::cout \u003c\u003c *(\u0026x);     // // print the value at the memory address\n\n   return 0;\n}\n```\n\n* and \u0026 essentially cancel each other out.\n\n\n---\n\n#### **Pointer** (type*): \n**object that holds a *memory address* of a variable.**\n\n```\nint;     // a normal int\nint\u0026;    // a reference to an int value\nint*;    // a pointer to an int value\n```\n\nIt is best practice to always initialize your pointers. \n\nOnce we have a pointer holding the address of another object, we can use the dereference operator (*) to access the value at that address. \n\nFor example,\n\n```\nint x = 5;\nint* ptr = \u0026x;\nstd::cout \u003c\u003c *ptr \u003c\u003c \"\\n\";    // prints 5\n```\n\n\n[See code here](./Fundamentals/C++/pointer.cpp)\n\n\n**Key insight**:\n\nWhen we use a pointer without a dereference (`ptr`), we are accessing the address held by the pointer. Modifying this (`ptr = \u0026y`) changes what the pointer is pointing at.\n\nWhen we dereference a pointer (`*ptr`), we are accessing the object being pointed at. Modifying this (`*ptr = 6;`) changes the value of the object being pointed at.\n\n## 2. Tensors\n\nIn tensor kernel development, you'll often use pointer arithmetic to:\n\n- Navigate through memory: Moving through tensor data efficiently\n- Implement strided access: Efficiently accessing non-contiguous elements\n- Handle different memory layouts: Working with row-major vs column-major data\n\nMemory layout for tensors refers to how tensor data is physically arranged in computer memory. Although tensors are multi-dimensional arrays, computer memory is linear (one-dimensional).\n\nThe memory layout is the mapping that determines how the multi-dimensional tensor indices translate to positions in linear memory.\n\n##### Contiguous Memory\nA tensor is stored contiguously when **all its elements are stored in a single, uninterrupted block of memory without gaps**.\n\n![Contiguous Memory](./assets/contiguous_memory.png)\n\n*shows a process requiring 3 memory blocks, being placed into a single continuous section of free memory in RAM.*\n\n##### Strides\n\nTake this piece of code:\n\n```\n# strides\nx = np.array([[1,2,3], \n              [4,5,6]], \n              dtype=np.int32)\n\nprint(x.strides)        # bytes to step in each dimension\n```\n\nThe output is (12, 4).\n\nEach element’s size in memory = number of bytes needed to store its value. \u003cbr\u003e\n`dtype=np.int32` means each element is a **32-bit** integer → **4 bytes** \u003cbr\u003e\n(32 bits ÷ 8 bits per byte = 4 bytes)\n\n- First number (12) → # of bytes to move **to the next row** (axis 0)\n    - each row has 3 elements x 4 bytes = 12 bytes\n- Second number (4) → bytes to move **to the next column** (axis 1)\n    - each step within a row moves by 1 element → 4 bytes\n\n![Strides](./assets/strides.png)\n*another example in int64*\n\n##### Intuition:\n\nThink of the array as **flattened memory**:\n\n`[1, 2, 3, 4, 5, 6]  ← all stored in a contiguous memory block`\n\n- from `1 → 2` → +4 bytes (column move)\n- from `1 → 4` → +12 bytes (row move)\n\n[See code here](./Fundamentals/Numpy/essentials.ipynb)\n\n\n## 3. Stack vs. heap allocation\n\n##### The heap\n\nThe heap segment (also known as the *\"free store\"*) keeps track of memory used for dynamic allocation.\n\nWhen a dynamically allocated variable is deleted, the memory is “returned” to the heap and can then be reassigned as future allocation requests are received. Remember that deleting a pointer does not delete the variable, it just returns the memory at the associated address back to the operating system.\n\n\n##### The (call) stack\n\nThe call stack keeps track of all the active functions (those that have been called but have not yet terminated) from the start of the program to the current point of execution, and handles allocation of all function parameters and local variables.\n\nFunctions are **pushed** on the stack when they are **called**, and **popped** when they **return**/finish.\n\n\n## Stack \u0026 heap in python memory management\n\nMemory in python is divided into two ideas: Stack memory and Heap memory.\n\n\n![Python Memory Management](./assets/python_mm.png)\n\n- Methods are executed in Stack memory. By default, the program executes `main()`\n- **Objects** are created in Heap\n- **References** are created in Stack\n\n- Once a function returns/finishes, it's place in Stack memory is deleted\n\n\n\u003cbr\u003e\n\n\n\n## Extra (Python) knowledge:\n\n- *Differentiate between Deep and Shallow Copies*\n\nA shallow copy creates a new object but inserts references to the original elements. Changes in nested objects affect both copies.\n\n`copy.copy(object)`\n\nA deep copy creates a new object and recursively copies all nested objects. Changes in nested objects are independent.\n\n`copy.deepcopy(object)`\n\nFor example,\n\n```\nimport copy\n\noriginal = [[1,2], [3,4]]\nshallow = copy.copy(original)\n\nshallow[0][0] = 999\n\nprint(original) # [[999, 2], [3, 4]]\nprint(shallow) # [[999, 2], [3, 4]]\n```\n\n`shallow` and `original` are different outer lists. But the inner lists (`[1,2]`, `[3, 4]`) are shared. So when we changed `shallow[0][0]`, it also changed in original.\n\n\n```\noriginal = [[1, 2], [3, 4]]\ndeep = copy.deepcopy(original)\n\ndeep[0][0] = 999\n\nprint(original)  # [[1, 2], [3, 4]]\nprint(deep)      # [[999, 2], [3, 4]]\n```\n\n`deepcopy` copied everything, including inner lists. So `original` and `deep` are now completely independent. Changing `deep` has no effect on `original`.\n\nIn summary, a shallow copy creates a **new outer object but reuses the inner (nested) objects**, so changes to nested parts affect both. For example, if you change the outer variable `1` in `shallow = [1, [2,3]]`, it **WON'T** change `original`. but if you change 2 or 3, it will. \n\nA deep copy creates a **new outer object and recursively copies all nested objects**, so changes are completely independent.\n\n\u003cbr\u003e\n\n- *Differentiate between List and Tuple*\n\nA list is mutable (can change after initialized) ex. `[1,2,3]`, while a tuple is immutable (cannot change once initialized) ex. `(1, 2, 3)`.\n\n\u003cbr\u003e\n\n\n\n\n\u003cbr\u003e\u003cbr\u003e\n\n**Additional resources:**\n- [What is the Difference Between a Pointer and a Reference C++](https://www.youtube.com/watch?v=sxHng1iufQE\u0026ab_channel=PaulProgramming)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdorsaroh%2Fkernels","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdorsaroh%2Fkernels","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdorsaroh%2Fkernels/lists"}