{"id":37099901,"url":"https://github.com/kubeservice-stack/cpusets-controller","last_synced_at":"2026-01-14T12:10:54.509Z","repository":{"id":65566650,"uuid":"582287463","full_name":"kubeservice-stack/cpusets-controller","owner":"kubeservice-stack","description":"cpusets-controller as a Device Plugin for Kubernetes, which exposes the CPU cores as consumable Devices to the Kubernetes scheduler.","archived":false,"fork":false,"pushed_at":"2023-07-16T14:44:17.000Z","size":7066,"stargazers_count":6,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-06-20T16:34:45.570Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kubeservice-stack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-26T10:44:13.000Z","updated_at":"2024-01-30T07:58:43.000Z","dependencies_parsed_at":"2024-06-20T16:19:55.282Z","dependency_job_id":"ea96ff52-614e-4884-a7ea-1993e45a098a","html_url":"https://github.com/kubeservice-stack/cpusets-controller","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/kubeservice-stack/cpusets-controller","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kubeservice-stack%2Fcpusets-controller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kubeservice-stack%2Fcpusets-controller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kubeservice-stack%2Fcpusets-controller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kubeservice-stack%2Fcpusets-controller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kubeservice-stack","download_url":"https://codeload.github.com/kubeservice-stack/cpusets-controller/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kubeservice-stack%2Fcpusets-controller/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28419598,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:47:48.104Z","status":"ssl_error","status_checked_at":"2026-01-14T10:46:19.031Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T12:10:53.811Z","updated_at":"2026-01-14T12:10:54.501Z","avatar_url":"https://github.com/kubeservice-stack.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 基于节点cpu精细化调度：cpusets-controller\n[![Docker publish](https://github.com/kubeservice-stack/cpusets-controller/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/kubeservice-stack/cpusets-controller/actions/workflows/ci.yml)\n[![Go Report Card](https://goreportcard.com/badge/github.com/kubeservice-stack/cpusets-controller)](https://goreportcard.com/report/github.com/kubeservice-stack/cpusets-controller)\n[![Go Reference](https://pkg.go.dev/badge/github.com/kubeservice-stack/lxcfs-webhook.svg)](https://pkg.go.dev/github.com/kubeservice-stack/lxcfs-webhook)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n\n**问：在 Kubernetes 中，运行多个集群节点是否存在隐藏成本？**\n答：是的，因为并非 Kubernetes 节点中的所有 `CPU` 和 `Memory` 都可用于运行 `Pod`。\n\n在一个 `Kubernetes 节点`中，`CPU` 和 `Memory` 分为：\n\n- 操作系统\n- Kubelet、CNI、CRI、CSI（+ 系统守护进程）\n- Pods\n- 驱逐门槛(Hard Eviction threshold)\n\n`Kubelet` 原生的`CPU Manager` 对于每一次`Pod`调度 `binding` 到 `Node` 时, 都会将计算 Node CPU是否够用\n\nNode预留CPU资源 reservedCPUs = KubeReservedCPUs + SystemReservedCPUs + HardEvictionThresholds ， 并向上取整，最终最为reserved cpus\n\n![](https://www.kubeservice.cn/img/k8s-kubelet/cpu-manager.png)\n\n虽然，对预留资源做了`限制`，但对于具体的`核没有做到绑核`运行。如果需要对Pod进行`绑核、亲核`部署 就再精细化实现了\n\n![](https://www.kubeservice.cn/img/k8s-kubelet/cpu-manager-reserved.png)\n\n\n## 场景\n\n如果需要对 `kubernetes` 中使用 `CPU 管理器` 进行如下更加精细化管理：\n![](https://www.kubeservice.cn/img/k8s-kubelet/cpu-pooler.png)\n\n- 解决在一个容器中可以同时使用独占cpu和共享cpu\n- 支持conatiner级别 core亲和 和 绑核\n- 兼容历史申请资源，优雅cgroup cpu驱逐\n- 独占CPU支持 绑核\n\n解决 传统应用`上云`，特定业务`绑核`运行 和 核心业务分级CPU亲和（`CPU Arrinity`）部署\n\n## 方案\n\n### 方案一：kubelet 的 cm上直接扩展\n直接在 `Kubelet`中，添加`CPUSets`对象中，添加`DeviceID` 和具体拓扑使用，并记录到`CheckPoints`文件中。\n\n优势：性能优，并且与`Kubelet预分配`资源做到协同\n缺点：\n- 1）更改原生代码，非云原生；\n- 2）`多Kubernetes`管理和升级复杂；\n- 3）`Kubernetes社区`难落地，抽象和迭代困难\n\n### 方案二：做云原生调度插件，替换/选用 `kubelet CPU Manager` 逻辑\n\n`Cpusets Controller` 提供一种 `Kubernetes` 的设备插件，将 `CPU` 内核作为可 `Device` 的 `Kubernetes` 调度程序.\n\n支持三种类型的 CPU 管理：\n- CPU独占\n- CPU共享\n- CPU默认（兼容默认cpu方式）\n\n![](https://www.kubeservice.cn/img/k8s-kubelet/cpuset.png)\n\n\n包含 `3` 个核心组件：\n\n- `device plugin`: `Kubernetes` 标准`Device`插件，将 CPU 池作为可调度资源无缝集成到 Kubernetes\n- `controller`: `Kubernetes` 标准 `Informer`，确保属于不同 `CPU Pooler` 的容器始终在物理上相互隔离管理和设置\n- `webhook`: 准入 Webhook验证, 校验 CPU 池特定的用户请求是否合法\n\n\n![](https://www.kubeservice.cn/img/k8s-kubelet/controller-cpuset.png)\n\n`Device Plugin` 的工作是通过现有的 `Device Plugin API` 将 `cpu分配` 作为`可消耗资源`注册给 `Kubelet`. `Device Plugin` 将 `CPU` 作为包含`物理CPU ID`列表的环境变量传递给容器。默认情况下，应用程序可以根据给定的 `CPU` 列表设置其进程 `CPU` 亲和力，或者可以将其留给标准的 `Linux Completely Fair Scheduler`。对于应用程序未实现设置其进程的 `CPU` 亲和力的功能的边缘情况，`CPU Pool`提供了代表应用程序设置它的机制。通过将应用程序进程信息配置到其 `Pod 规范`的注释字段来启用此选择加入功能。\n\n`Webhook`: 以根据启动二进制文件（安装、环境变量等）的需要来改变 Pod 的规范。\n\n`Controller` 子组件通过 `Linux cpusets` 实现容器的完全物理分离。通过 `Informer` 不断地监视 `Kubernetes` 的 `Pod API`，并在创建 Pod 或更改其状态（例如重新启动等）时触发。 `shared` 在共享的情况下，或者默认情况下容器没有明确要求任何池化资源。`Controller` 然后将计算出的集合提供给容器的 `cgroupfs` 文件系统 (`cpuset.cpus`) 。\n\n优势：\n- 1)对CPU进一步池化，实现部分绑核能力；\n- 2)对现有的部署不影响\n- 3)特定绑核使用，使大颗粒高性能服务可上云；\n\n缺点：\n- 1)default 与 shared共享问题\n- 2)Kubelet CPU Manager 与 CPUSet Controller协同问题\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkubeservice-stack%2Fcpusets-controller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkubeservice-stack%2Fcpusets-controller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkubeservice-stack%2Fcpusets-controller/lists"}