{"id":31806775,"url":"https://github.com/multikernel/kerf","last_synced_at":"2025-10-11T03:49:59.054Z","repository":{"id":318587774,"uuid":"1071918700","full_name":"multikernel/kerf","owner":"multikernel","description":"kerf is a comprehensive multikernel management system designed to orchestrate and manage multiple kernel instances on a single host.","archived":false,"fork":false,"pushed_at":"2025-10-08T02:28:21.000Z","size":46,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-08T04:17:18.995Z","etag":null,"topics":["device-tree","linux-kernel","multikernel"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/multikernel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-08T02:14:58.000Z","updated_at":"2025-10-08T03:12:40.000Z","dependencies_parsed_at":"2025-10-08T04:17:20.588Z","dependency_job_id":"58443f39-6e97-40b7-99f1-7a3da6b1f945","html_url":"https://github.com/multikernel/kerf","commit_stats":null,"previous_names":["multikernel/kerf"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/multikernel/kerf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/multikernel%2Fkerf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/multikernel%2Fkerf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/multikernel%2Fkerf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/multikernel%2Fkerf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/multikernel","download_url":"https://codeload.github.com/multikernel/kerf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/multikernel%2Fkerf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006108,"owners_count":26084026,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["device-tree","linux-kernel","multikernel"],"created_at":"2025-10-11T03:49:57.962Z","updated_at":"2025-10-11T03:49:59.045Z","avatar_url":"https://github.com/multikernel.png","language":"Python","readme":"# Kerf: Multikernel Management System\n\n## Overview\n\n`kerf` is a comprehensive multikernel management system designed to orchestrate and manage multiple kernel instances on a single host. Starting with advanced device tree compilation and validation, `kerf` provides the foundation for complete multikernel lifecycle management.\n\nUnlike standard tools that only perform basic format conversion, `kerf` understands multikernel semantics and **always validates** resource allocations, detects conflicts, and extracts instance-specific device trees from global configurations. The system is architected to evolve into a complete multikernel runtime environment.\n\n## Vision \u0026 Roadmap\n\n### Current Phase: Device Tree Foundation\n`kerf` currently provides the essential device tree compilation and validation capabilities needed for multikernel systems:\n\n1. **Resource Conflict Detection**: Multiple instances might accidentally be allocated the same CPUs, overlapping memory regions, or the same devices\n2. **Over-Allocation**: The sum of all allocations might exceed available resources\n3. **Invalid References**: Instances might reference non-existent hardware or devices reserved for the host\n4. **Atomicity**: All allocations should be validated together before deployment\n5. **Instance Extraction**: Each spawned kernel needs a device tree showing only its allocated resources, not the entire system\n\n### Future Phases: Complete Multikernel Runtime\nThe `kerf` system is designed to evolve into a comprehensive multikernel management platform:\n\n- **Kernel Loading \u0026 Execution**: Load and execute multiple kernel instances with proper isolation\n- **Resource Management**: Dynamic allocation and deallocation of system resources\n- **Instance Lifecycle**: Start, stop, pause, and migrate kernel instances\n- **Monitoring \u0026 Debugging**: Real-time monitoring of kernel instances and system health\n- **Security \u0026 Isolation**: Advanced security policies and isolation mechanisms\n- **Orchestration**: High-level orchestration of complex multikernel workloads\n\nThe current device tree foundation provides the critical infrastructure needed for these advanced capabilities.\n\n## Architecture\n\n### Design Philosophy\n\nThe `kerf` system is built on foundational principles that support both current device tree capabilities and future multikernel runtime features:\n\n1. **Single Source of Truth**: One global DTS describes the entire system - hardware inventory, host reservations, and all spawn kernel allocations\n2. **Multiple Binary Outputs**: kerf compiles one DTS into multiple DTB files (global + per-instance)\n3. **Mandatory Validation**: Every operation validates the configuration - validation is not optional\n4. **Fail-Fast**: Catch resource conflicts immediately, never produce invalid output\n5. **Instance Isolation**: Generate minimal, instance-specific device trees that show only what each kernel should see\n6. **Extensible Architecture**: Designed to support future kernel loading, execution, and management capabilities\n7. **Developer-Friendly**: Clear error messages with suggestions for fixing problems\n8. **Runtime-Ready**: Current design anticipates future kernel execution and lifecycle management needs\n\n### Compilation Model\n\n```\nInput: Single Global DTS\n         │\n         ▼\n    ┌─────────┐\n    │ kerf    │ ← Always validates\n    │ Compiler│\n    └─────────┘\n         │\n         ├──────────────┬──────────────┬──────────────┐\n         ▼              ▼              ▼              ▼\n    global.dtb   instance1.dtb   instance2.dtb   instance3.dtb\n    (complete)   (minimal)       (minimal)       (minimal)\n```\n\n**Key Points:**\n- **One DTS input** describing entire system\n- **Multiple DTB outputs** (1 global + N instance-specific)\n- **Validation happens once** during compilation\n- **All outputs are pre-validated** and guaranteed consistent\n\n## Current Capabilities\n\n### Device Tree Compilation \u0026 Validation\n- **Advanced Validation**: Comprehensive resource conflict detection and validation\n- **Instance Extraction**: Generate minimal, instance-specific device trees\n- **Format Support**: DTS to DTB compilation with multiple output formats\n- **Error Reporting**: Detailed error messages with actionable suggestions\n- **Resource Analysis**: Complete resource utilization reporting\n- **CPU \u0026 NUMA Topology**: Full support for CPU topology and NUMA-aware resource allocation\n\n### Command Line Interface\n```bash\n# Compile and validate system configuration\nkerf dtc --input=system.dts --output-dir=build/\n\n# Extract specific kernel instance\nkerf dtc --input=global.dtb --extract=web-server --output=web-server.dtb\n\n# Generate validation reports\nkerf dtc --input=system.dts --report --verbose\n```\n\n### Modular Architecture\nThe `kerf` system is designed with a modular architecture that supports incremental development:\n\n- **`kerf dtc`**: Device tree compilation and validation (current)\n- **`kerf load`**: Kernel loading (future)\n- **`kerf exec`**: Kernel execuation (future)\n- **`kerf update`**: Update a kernel instawnce (future)\n- **`kerf kill`**: Kill a kernel instance (future)\n\nThis modular design allows users to adopt `kerf` incrementally, starting with device tree validation and expanding to full multikernel management as features become available.\n\n### Technical Foundation\nThe current device tree foundation provides essential building blocks for future multikernel capabilities:\n\n- **Resource Validation**: Ensures safe resource allocation before kernel execution\n- **Instance Isolation**: Provides the foundation for secure kernel isolation\n- **Configuration Management**: Enables consistent and validated system configurations\n- **Error Handling**: Establishes patterns for robust error reporting and recovery\n- **Extensible Architecture**: Designed to support future kernel management APIs\n\nThese foundational capabilities are essential for safe and reliable multikernel execution, making `kerf` the ideal platform for building comprehensive multikernel management systems.\n\n## Future Roadmap\n\n### Phase 2: Kernel Loading \u0026 Execution\n- **Kernel Image Management**: Load and manage multiple kernel images\n- **Instance Boot**: Start kernel instances with validated device trees\n- **Resource Binding**: Bind allocated resources to running instances\n- **Instance Monitoring**: Basic health monitoring of running instances\n\n### Phase 3: Advanced Management\n- **Dynamic Resource Allocation**: Runtime resource reallocation\n- **Instance Migration**: Move instances between hosts\n- **Advanced Security**: Enhanced isolation and security policies\n- **Orchestration APIs**: High-level management interfaces\n\n## Global Device Tree Format\n\n### Structure Overview\n\nThe global device tree contains three main sections that map directly to the sysfs hierarchy:\n\n1. **Resources** (`/resources`): Complete description of all physical resources\n2. **Instances** (`/instances`): Resource assignments for each spawn kernel\n3. **Device References**: Linkage between instances and hardware devices\n\n### Complete Example\n\n```dts\n/multikernel-v1/;\n\n/ {\n    compatible = \"linux,multikernel-host\";\n    \n    // ========== MAPS TO /sys/kernel/multikernel/device_tree ==========\n    resources {\n        cpus {\n            total = \u003c32\u003e;\n            host-reserved = \u003c0 1 2 3\u003e;\n            available = \u003c4 5 6 7 8 9 10 11 12 13 14 15 \n                        16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31\u003e;\n        };\n        \n        memory {\n            total-bytes = \u003c0x0 0x400000000\u003e;      // 16GB\n            host-reserved-bytes = \u003c0x0 0x80000000\u003e; // 2GB\n            memory-pool-base = \u003c0x80000000\u003e;\n            memory-pool-bytes = \u003c0x0 0x380000000\u003e;  // 14GB\n        };\n        \n        devices {\n            eth0: ethernet@0 {\n                compatible = \"intel,i40e\";\n                pci-id = \"0000:01:00.0\";\n                sriov-vfs = \u003c8\u003e;\n                host-reserved-vf = \u003c0\u003e;\n                available-vfs = \u003c1 2 3 4 5 6 7\u003e;\n            };\n            \n            nvme0: storage@0 {\n                compatible = \"nvme\";\n                pci-id = \"0000:02:00.0\";\n                namespaces = \u003c4\u003e;\n                host-reserved-ns = \u003c1\u003e;\n                available-ns = \u003c2 3 4\u003e;\n            };\n        };\n    };\n    \n    // ========== MAPS TO /sys/kernel/multikernel/instances/ ==========\n    instances {\n        // Maps to /sys/kernel/multikernel/instances/web-server/\n        web-server {\n            id = \u003c1\u003e;\n            \n            resources {\n                cpus = \u003c4 5 6 7\u003e;\n                memory-base = \u003c0x80000000\u003e;\n                memory-bytes = \u003c0x80000000\u003e;  // 2GB\n                devices = \u003c\u0026eth0_vf1\u003e;\n            };\n            \n        };\n        \n        // Maps to /sys/kernel/multikernel/instances/database/\n        database {\n            id = \u003c2\u003e;\n            \n            resources {\n                cpus = \u003c8 9 10 11 12 13 14 15\u003e;\n                memory-base = \u003c0x100000000\u003e;\n                memory-bytes = \u003c0x200000000\u003e;  // 8GB\n                devices = \u003c\u0026eth0_vf2\u003e, \u003c\u0026nvme0_ns2\u003e;\n            };\n            \n        };\n        \n        // Maps to /sys/kernel/multikernel/instances/compute/\n        compute {\n            id = \u003c3\u003e;\n            \n            resources {\n                cpus = \u003c16 17 18 19 20 21 22 23\u003e;\n                memory-base = \u003c0x300000000\u003e;\n                memory-bytes = \u003c0x100000000\u003e;  // 4GB\n            };\n            \n        };\n    };\n    \n    // ========== DEVICE REFERENCES (phandle targets) ==========\n    eth0_vf1: ethernet-vf@1 {\n        parent = \u003c\u0026eth0\u003e;\n        vf-id = \u003c1\u003e;\n    };\n    \n    eth0_vf2: ethernet-vf@2 {\n        parent = \u003c\u0026eth0\u003e;\n        vf-id = \u003c2\u003e;\n    };\n    \n    nvme0_ns2: nvme-ns@2 {\n        parent = \u003c\u0026nvme0\u003e;\n        namespace-id = \u003c2\u003e;\n    };\n};\n```\n\n### Mapping to Sysfs Hierarchy\n\n**Device Tree Structure → Sysfs Hierarchy:**\n\n```\nDTS: /resources/cpus                →  /sys/kernel/multikernel/resources/cpus/\nDTS: /resources/memory              →  /sys/kernel/multikernel/resources/memory/\nDTS: /resources/devices             →  /sys/kernel/multikernel/resources/devices/\nDTS: /instances/web-server           →  /sys/kernel/multikernel/instances/web-server/\nDTS: /instances/database             →  /sys/kernel/multikernel/instances/database/\n```\n\n**Name-based addressing:**\n- Instance node name in DTS (`web-server`) = directory name in sysfs (`/instances/web-server/`)\n- Kernel assigns numeric IDs, but users reference by name\n- No manual ID coordination needed\n\n## Validation Rules\n\n### Validation is Mandatory\n\n**All `kerf` operations perform validation automatically:**\n- Compiling DTS to DTB → validates\n- Extracting an instance → validates first\n- Converting formats → validates\n- Generating reports → validates first\n\n**Validation cannot be disabled or skipped.**\n\n### CPU Allocation Validation\n\n**Rules:**\n1. All CPUs must exist in hardware inventory (0 to `total-1`)\n2. CPUs must be in the `available` list (not `host-reserved`)\n3. No CPU can be allocated to multiple instances\n4. CPU lists should be explicitly enumerated\n\n**Error Examples:**\n```\nERROR: Instance database: CPU 35 does not exist (hardware has 0-31)\nERROR: Instance web-server: CPU 2 is reserved for host kernel\nERROR: Instance web-server and compute: CPU overlap detected (CPUs 8-11)\n```\n\n### Memory Allocation Validation\n\n**Rules:**\n1. All memory regions must be within memory pool bounds\n2. Memory regions cannot overlap between instances\n3. Sum of all allocations must not exceed memory pool size\n4. Memory base addresses must be page-aligned (4KB = 0x1000)\n\n**Error Examples:**\n```\nERROR: Instance database: Memory [0x50000000-0x60000000] outside memory pool\nERROR: Instance web-server and database: Memory overlap [0x100000000-0x120000000]\nERROR: Total memory allocation (16GB) exceeds memory pool (14GB)\nWARNING: Instance compute: Memory base 0x80000001 not page-aligned\n```\n\n### Device Allocation Validation\n\n**Rules:**\n1. Referenced devices must exist in hardware inventory\n2. Devices can only be allocated to one instance (exclusive access)\n3. Device references must be valid (no dangling phandles)\n4. SR-IOV VF numbers must be within available range\n5. Namespace IDs must be within available range\n\n**Error Examples:**\n```\nERROR: Instance database: Reference to non-existent device 'nvme1'\nERROR: Instance web-server and compute: Both allocated eth0:vf2\nERROR: Instance database: VF ID 10 exceeds available VFs (1-7)\nERROR: Instance database: Namespace 1 is reserved for host kernel\n```\n\n### Global Resource Validation\n\n**Rules:**\n1. Instance names must be unique\n2. Instance IDs must be unique\n3. All phandle references must resolve\n4. Hardware inventory must be complete and consistent\n\n**Error Examples:**\n```\nERROR: Duplicate instance name: \"web-server\" appears twice\nERROR: Duplicate instance ID: 2 assigned to both database and compute\nERROR: Dangling phandle reference: eth0_vf99 not defined\nWARNING: 12 CPUs (43% of memory pool) are unallocated\n```\n\n## Instance Extraction\n\n### Purpose\n\nFrom a global device tree containing all system information, `kerf` extracts a minimal device tree for each spawn kernel instance that contains:\n- Only the CPUs allocated to that instance\n- Only the memory region assigned to that instance\n- Only the devices accessible by that instance\n- Configuration and build hints specific to that instance\n\n### Extraction Process\n\n1. **Validate Global DTS/DTB**: Mandatory validation before extraction\n2. **Locate Instance Node**: Find `/instances/{name}` node\n3. **Extract Resources**: Read CPU, memory, and device allocations\n4. **Build Minimal DTB**: Create new device tree with:\n   - `/chosen` node with resource assignments\n   - Device nodes for allocated hardware only\n   - Configuration hints from instance config\n5. **Generate Binary**: Output instance-specific DTB file\n\n### Instance DTB Structure\n\n**Example: web-server.dtb (extracted from global)**\n\n```dts\n/multikernel-v1/;\n\n/ {\n    compatible = \"linux,multikernel-instance\";\n    \n    chosen {\n        // Resource assignments for this instance only\n        linux,multikernel-cpus = \u003c4 5 6 7\u003e;\n        linux,multikernel-memory-base = \u003c0x80000000\u003e;\n        linux,multikernel-memory-size = \u003c0x80000000\u003e;\n        linux,multikernel-instance-id = \u003c1\u003e;\n        linux,multikernel-instance-name = \"web-server\";\n    };\n    \n    // Only devices allocated to this instance\n    ethernet@0 {\n        compatible = \"intel,i40e\";\n        reg = \u003c0x0 0x1000\u003e;\n        vf-id = \u003c1\u003e;  // This is VF1 assigned to web-server\n    };\n    \n};\n```\n\n**Key characteristics:**\n- Contains **only** resources for this instance\n- No knowledge of other instances\n- No global hardware inventory\n- Minimal and focused\n\n## Command-Line Interface\n\n### Basic Commands\n\n```bash\n# Compile DTS to global DTB (validates automatically)\nkerf dtc --input=system.dts --output=global.dtb\n\n# Compile DTS to global DTB and extract all instances\nkerf dtc --input=system.dts --output-dir=build/\n# Generates:\n#   build/global.dtb\n#   build/web-server.dtb\n#   build/database.dtb\n#   build/compute.dtb\n\n# Extract single instance from global DTB\nkerf dtc --input=global.dtb --extract=web-server --output=web-server.dtb\n\n# Extract all instances from global DTB\nkerf dtc --input=global.dtb --extract-all --output-dir=instances/\n\n# Generate allocation report\nkerf dtc --input=global.dtb --report\n\n# Convert DTB back to DTS\nkerf dtc --input=global.dtb --output=global.dts --format=dts\n```\n\n### Advanced Commands\n\n```bash\n# Verbose output (shows validation details)\nkerf dtc --input=system.dts --output=global.dtb --verbose\n\n# Dry-run: validate and show what would be generated\nkerf dtc --input=system.dts --dry-run\n\n# Extract specific instance by name (not ID)\nkerf dtc --input=global.dtb --extract=database --output=db.dtb\n\n# List all instances in global DTB\nkerf dtc --input=global.dtb --list-instances\n# Output:\n# web-server (ID: 1)\n# database (ID: 2)\n# compute (ID: 3)\n```\n\n### Output Formats\n\n```bash\n# Human-readable text (default)\nkerf dtc --input=global.dtb --report\n\n# JSON for tooling integration\nkerf dtc --input=global.dtb --report --format=json\n\n# YAML for configuration management\nkerf dtc --input=global.dtb --report --format=yaml\n\n# DTS (human-readable device tree source)\nkerf dtc --input=global.dtb --output=global.dts --format=dts\n```\n\n## Integration with Kernel\n\n### Sysfs Interface\n\nThe kernel exposes a sysfs interface that mirrors the device tree structure:\n\n```\n/sys/kernel/multikernel/\n├── device_tree              # Read/Write: Global DTB\n├── device_tree_source       # Read-only: Global DTS (human-readable)\n│\n└── instances/               # Auto-generated from /instances in DTB\n    ├── web-server/         # Directory name from DTS node name\n    │   ├── id              # Read-only: \"1\"\n    │   ├── device_tree     # Read-only: Instance-specific DTB\n    │   ├── device_tree_source  # Read-only: Instance DTS\n    │   ├── status          # Read-only: \"ready\", \"active\", \"stopped\"\n    │\n    ├── database/\n    │   └── ...\n    │\n    └── compute/\n        └── ...\n```\n\n### Workflow: Initial Setup\n\n```bash\n# Step 1: Write global DTS describing entire system\nvim system.dts\n\n# Step 2: Compile and validate with kerf dtc\nkerf dtc --input=system.dts --output-dir=build/\n# Output:\n#   ✓ Validation passed\n#   Generated: build/global.dtb\n#   Generated: build/web-server.dtb\n#   Generated: build/database.dtb\n#   Generated: build/compute.dtb\n\n# Step 3: Upload global DTB to kernel\ncat build/global.dtb \u003e /sys/kernel/multikernel/device_tree\n\n# Kernel automatically:\n# - Validates global DTB (defense-in-depth)\n# - Creates /sys/kernel/multikernel/instances/{web-server,database,compute}/\n# - Generates instance-specific DTBs internally\n# - Populates /sys/kernel/multikernel/resources/\n\n# Step 4: Verify instance creation\nls /sys/kernel/multikernel/instances/\n# web-server  database  compute\n\n# Step 5: View instance configuration\ncat /sys/kernel/multikernel/instances/web-server/device_tree_source\n# Shows instance-specific DTS (only web-server resources)\n\n# Step 6: Load kernel with instance DTB (use kerf dtc generated or kernel generated)\nkexec_file_load(/boot/vmlinuz, build/web-server.dtb, KEXEC_MULTIKERNEL | KEXEC_MK_ID(1))\n# Or use kernel-generated:\n# kexec_file_load(/boot/vmlinuz, /sys/kernel/multikernel/instances/web-server/device_tree, ...)\n```\n\n### Workflow: Dynamic Updates\n\n```bash\n# Step 1: Modify global DTS\nvim system.dts\n# Example: Change database CPUs from 8-15 to 8-19 (add 4 CPUs)\n\n# Step 2: Recompile and validate\nkerf dtc --input=system.dts --output-dir=build/\n# Output:\n#   ✓ Validation passed\n#   Generated: build/global.dtb (updated)\n#   Generated: build/database.dtb (updated - now has CPUs 8-19)\n#   (other instances unchanged)\n\n# Step 3: Upload updated global DTB\ncat build/global.dtb \u003e /sys/kernel/multikernel/device_tree\n\n# Kernel automatically:\n# - Validates new global DTB\n# - Calculates resource deltas for all instances\n# - Phase 1: Releases resources being removed\n# - Phase 2: Allocates new resources\n# - Updates instance DTBs in sysfs\n# - Notifies spawned kernels via shared memory/interrupts\n\n# Step 4: Verify update\ncat /sys/kernel/multikernel/instances/database/device_tree_source | grep cpus\n# linux,multikernel-cpus = \u003c8 9 10 11 12 13 14 15 16 17 18 19\u003e;\n```\n\n\n## Validation Output Examples\n\n### Successful Validation\n\n```\n$ kerf dtc --input=system.dts --output-dir=build/\n\nMultikernel Device Tree Validation Report\n==========================================\nStatus: ✓ VALID\n\nHardware Inventory:\n  CPUs: 32 total\n    Host reserved: 0-3 (4 CPUs, 12%)\n    Memory pool: 4-31 (28 CPUs, 88%)\n  Memory: 16GB total\n    Host reserved: 2GB (12%)\n    Memory pool: 14GB at 0x80000000 (88%)\n  Devices: 2 network, 1 storage\n\nInstance Allocations:\n  web-server (ID: 1):\n    CPUs: 4-7 (4 CPUs, 14% of pool)\n    Memory: 2GB at 0x80000000 (14% of pool)\n    Devices: eth0:vf1\n    Status: ✓ Valid\n    \n  database (ID: 2):\n    CPUs: 8-15 (8 CPUs, 29% of pool)\n    Memory: 8GB at 0x100000000 (57% of pool)\n    Devices: eth0:vf2, nvme0:ns2\n    Status: ✓ Valid\n    \n  compute (ID: 3):\n    CPUs: 16-23 (8 CPUs, 29% of pool)\n    Memory: 4GB at 0x300000000 (29% of pool)\n    Devices: none\n    Status: ✓ Valid\n\nResource Utilization:\n  CPUs: 20/28 allocated (71%), 8 free\n  Memory: 14/14 GB allocated (100%), 0 free\n  Network: 2/7 VFs allocated (29%)\n  Storage: 1/3 namespaces allocated (33%)\n  \n✓ All validations passed\n\nGenerated output:\n  build/global.dtb (3847 bytes)\n  build/web-server.dtb (1024 bytes)\n  build/database.dtb (1536 bytes)\n  build/compute.dtb (896 bytes)\n```\n\n### Failed Validation\n\n```\n$ kerf dtc --input=bad_system.dts --output=global.dtb\n\nMultikernel Device Tree Validation Report\n==========================================\nStatus: ✗ INVALID\n\nERROR: Instance database: CPU allocation conflict with web-server\n  web-server uses CPUs: 4-11\n  database requested CPUs: 8-15\n  Overlapping CPUs: 8, 9, 10, 11\n  \n  Suggestion: Change database to use CPUs 12-19\n  Alternative: Reduce web-server to CPUs 4-7\n  \n  In file system.dts:\n    Line 45: web-server { cpus = \u003c4 5 6 7 8 9 10 11\u003e; }\n    Line 68: database { cpus = \u003c8 9 10 11 12 13 14 15\u003e; }\n\nERROR: Instance compute: Memory allocation exceeds memory pool\n  Memory pool: 0x80000000 - 0x400000000 (14GB available)\n  compute requested: 0x400000000 - 0x500000000 (4GB)\n  Exceeds pool end by: 4GB\n  \n  Suggestion: Change memory-base to 0x300000000\n  Note: Requires reducing other instance allocations\n  \n  In file system.dts:\n    Line 89: memory-base = \u003c0x400000000\u003e;\n\nWARNING: Resource utilization\n  12 CPUs (43% of memory pool) remain unallocated\n  Consider: Allocate remaining CPUs or reduce total instances\n\n✗ Validation failed with 2 errors, 1 warning\nNo output generated - fix errors first\n\nExit code: 1\n```\n\n## Error Messages and Suggestions\n\n### Design Principles\n\nError messages should be:\n1. **Clear**: Explain what's wrong in simple terms\n2. **Actionable**: Suggest how to fix the problem\n3. **Contextual**: Show relevant configuration and file locations\n4. **Non-judgmental**: Help, don't blame\n5. **Educational**: Help users understand multikernel constraints\n\n### Error Message Format\n\n```\nERROR: \u003cInstance\u003e: \u003cProblem Category\u003e\n  \u003cDetailed explanation\u003e\n  Current state: \u003cWhat is currently configured\u003e\n  Conflict/Issue: \u003cWhat's wrong with it\u003e\n  \n  Suggestion: \u003cPrimary fix recommendation\u003e\n  Alternative: \u003cAlternative fix if applicable\u003e\n  \n  In file \u003cfilename\u003e:\n    Line \u003cN\u003e: \u003crelevant source line\u003e\n```\n\n### Detailed Error Examples\n\n**CPU Overlap Error:**\n```\nERROR: database: CPU allocation conflict with web-server\n  Instance web-server uses CPUs: 4-11\n  Instance database requested CPUs: 8-15\n  Overlapping CPUs: 8, 9, 10, 11\n  \n  Suggestion: Change database to use CPUs 12-19\n  Alternative: Reduce web-server allocation to CPUs 4-7\n  \n  In file system.dts:\n    Line 35: web-server { cpus = \u003c4 5 6 7 8 9 10 11\u003e; }\n    Line 58: database { cpus = \u003c8 9 10 11 12 13 14 15\u003e; }\n```\n\n**Memory Overflow Error:**\n```\nERROR: compute: Memory allocation exceeds memory pool\n  Memory pool range: 0x80000000 - 0x400000000 (14GB available)\n  Instance compute requested: 0x400000000 - 0x500000000 (4GB)\n  Overflow: Exceeds pool end by 4GB (0x100000000 bytes)\n  \n  Suggestion: Change memory-base to 0x300000000\n  Note: This would place memory at end of memory pool\n  \n  In file system.dts:\n    Line 78: memory-base = \u003c0x400000000\u003e;\n    \n  Context:\n    Memory pool ends at: 0x400000000\n    Requested start: 0x400000000 (exactly at pool end)\n    Requested size: 0x100000000\n    Would end at: 0x500000000 (outside pool)\n```\n\n**Device Not Found Error:**\n```\nERROR: database: Invalid device reference\n  Requested device: nvme1:ns1\n  Available storage devices: nvme0\n  Available namespaces on nvme0: ns2, ns3, ns4\n  Note: Namespace ns1 is reserved for host kernel\n  \n  Suggestion: Change device reference to nvme0:ns2\n  \n  In file system.dts:\n    Line 64: devices = \u003c\u0026nvme1_ns1\u003e;\n    \n  Did you mean:\n    - nvme0:ns2 (available)\n    - nvme0:ns3 (available)\n    - nvme0:ns4 (available)\n```\n\n**Memory Overlap Error:**\n```\nERROR: database and compute: Memory region overlap detected\n  database memory: 0x100000000 - 0x300000000 (8GB)\n  compute memory:  0x280000000 - 0x380000000 (4GB)\n  Overlapping region: 0x280000000 - 0x300000000 (2GB overlap)\n  \n  Suggestion: Move compute memory to 0x300000000\n  Alternative: Reduce database memory size to 6GB (end at 0x280000000)\n  \n  In file system.dts:\n    Line 62: database { memory-base = \u003c0x100000000\u003e; memory-bytes = \u003c0x200000000\u003e; }\n    Line 82: compute { memory-base = \u003c0x280000000\u003e; memory-bytes = \u003c0x100000000\u003e; }\n```\n\n**Duplicate Instance Name Error:**\n```\nERROR: Duplicate instance name: \"web-server\"\n  Instance name \"web-server\" appears multiple times in configuration\n  Found at:\n    Line 28: instances { web-server { id = \u003c1\u003e; ... } }\n    Line 95: instances { web-server { id = \u003c4\u003e; ... } }\n  \n  Suggestion: Rename the second instance to \"web-server-2\" or another unique name\n  Note: Instance names must be unique across the entire system\n```\n\n**Misaligned Memory Error:**\n```\nWARNING: web-server: Memory base address not page-aligned\n  Requested base: 0x80000001\n  Page size: 4KB (0x1000)\n  Alignment requirement: Address must be multiple of 0x1000\n  \n  Suggestion: Use base address 0x80000000 (already aligned)\n  Note: Misaligned addresses may cause performance issues or boot failures\n  \n  In file system.dts:\n    Line 38: memory-base = \u003c0x80000001\u003e;\n```\n\n\n## Dependencies\n\n### Required Dependencies\n\n```toml\n[tool.poetry.dependencies]\npython = \"^3.8\"\npylibfdt = \"^1.7.0\"      # Device tree parsing (from dtc project)\n```\n\n### Installation\n\n```bash\n# From PyPI (future)\npip install kerf\n\n# From source (development)\ngit clone https://github.com/multikernel/kerf.git\ncd kerf\npip install -e \".[dev,cli,formats]\"\n\n# System packages (future)\napt install kerf              # Debian/Ubuntu\ndnf install kerf              # Fedora/RHEL\npacman -S kerf                # Arch Linux\n```\n\n### Getting Started\n\n```bash\n# Install in development mode\npip install -e .\n\n# Test the installation\nkerf --help\nkerf dtc --help\n\n# Try with example configuration\nkerf dtc --input=examples/system.dts --output-dir=build/\n```\n\n## Examples\n\nThe `examples/` directory contains sample Device Tree Source (DTS) files demonstrating various multikernel configurations:\n\n- **`system.dts`** - Complete multikernel system with web server, database, and compute instances (32 CPUs, 16GB memory)\n- **`minimal.dts`** - Simple configuration for testing and development (8 CPUs, 8GB memory)\n- **`high_performance.dts`** - Large-scale configuration for high-performance computing (128 CPUs, 64GB memory)\n- **`edge_computing.dts`** - Edge computing configuration with GPU support for AI inference (16 CPUs, 32GB memory)\n- **`numa_topology.dts`** - Advanced NUMA topology configuration with 4 NUMA nodes and topology-aware allocation\n- **`simple_numa.dts`** - Basic NUMA configuration with 2 NUMA nodes and NUMA-aware policies\n- **`conflict_example.dts`** - Intentionally invalid configuration demonstrating common validation errors\n- **`bad_system.dts`** - Another example of invalid configuration for testing error detection\n\n## CPU and NUMA Topology Support\n\nKerf provides comprehensive support for CPU and NUMA topology management:\n\n### Key Features\n- **CPU Topology**: Socket, core, and thread mapping with SMT/hyperthreading support\n- **NUMA Awareness**: NUMA node definition with memory regions and CPU assignments\n- **Topology Policies**: CPU affinity (`compact`, `spread`, `local`) and memory policies (`local`, `interleave`, `bind`)\n- **Performance Validation**: Automatic validation of topology constraints and performance warnings\n\n### Example NUMA Configuration\n```dts\nresources {\n    cpus {\n        total = \u003c32\u003e;\n        host-reserved = \u003c0 1 2 3\u003e;\n        available = \u003c4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 \n                    20 21 22 23 24 25 26 27 28 29 30 31\u003e;\n    };\n    \n    topology {\n        numa-nodes {\n            node@0 {\n                node-id = \u003c0\u003e;\n                memory-base = \u003c0x0 0x0\u003e;\n                memory-size = \u003c0x0 0x800000000\u003e;  // 16GB\n                cpus = \u003c0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15\u003e;\n            };\n            \n            node@1 {\n                node-id = \u003c1\u003e;\n                memory-base = \u003c0x0 0x800000000\u003e;\n                memory-size = \u003c0x0 0x800000000\u003e;  // 16GB\n                cpus = \u003c16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31\u003e;\n            };\n        };\n    };\n};\n\ninstances {\n    web-server {\n        resources {\n            cpus = \u003c4 5 6 7 8 9 10 11\u003e;  // NUMA node 0\n            memory-base = \u003c0x0 0x800000000\u003e;\n            memory-bytes = \u003c0x0 0x200000000\u003e;  // 8GB\n            numa-nodes = \u003c0\u003e;\n            cpu-affinity = \"compact\";\n            memory-policy = \"local\";\n        };\n    };\n};\n```\n\nFor detailed information about CPU and NUMA topology support, see [CPU_NUMA_TOPOLOGY.md](docs/CPU_NUMA_TOPOLOGY.md).\n\n## References\n\n- **Device Tree Specification**: https://devicetree-specification.readthedocs.io/\n- **libfdt Documentation**: https://git.kernel.org/pub/scm/utils/dtc/dtc.git/tree/Documentation\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmultikernel%2Fkerf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmultikernel%2Fkerf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmultikernel%2Fkerf/lists"}