{"id":13839355,"url":"https://github.com/cwdsuzhou/super-scheduling","last_synced_at":"2025-04-15T02:36:33.392Z","repository":{"id":127979305,"uuid":"385847799","full_name":"cwdsuzhou/super-scheduling","owner":"cwdsuzhou","description":"a topology-scheduler and a descheduler extened from descheduler.","archived":false,"fork":false,"pushed_at":"2021-09-03T02:30:50.000Z","size":130,"stargazers_count":15,"open_issues_count":0,"forks_count":8,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-28T14:43:14.355Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cwdsuzhou.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-07-14T07:11:04.000Z","updated_at":"2024-08-05T02:08:00.000Z","dependencies_parsed_at":"2024-01-20T21:46:32.818Z","dependency_job_id":"4e77b25c-075c-42cf-b1e1-3055202b6629","html_url":"https://github.com/cwdsuzhou/super-scheduling","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwdsuzhou%2Fsuper-scheduling","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwdsuzhou%2Fsuper-scheduling/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwdsuzhou%2Fsuper-scheduling/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwdsuzhou%2Fsuper-scheduling/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cwdsuzhou","download_url":"https://codeload.github.com/cwdsuzhou/super-scheduling/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248995100,"owners_count":21195497,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T17:00:20.087Z","updated_at":"2025-04-15T02:36:32.882Z","avatar_url":"https://github.com/cwdsuzhou.png","language":"Go","funding_links":[],"categories":["Scheduler"],"sub_categories":[],"readme":"# Super scheduling\n\n## Introduction\n\nThis project includes a topology-scheduler and a descheduler extened\nfrom [descheduler](https://github.com/kubernetes-sigs/descheduler.git).\n\ntopology-scheduler will help scheduling pods cross zones, regions or clusters.\n\n\u003e We would like this project be merged by upstream in the future, so crd and codes includes xxx.scheduling.sigs.k8s.io\n\n## Why we need this\n\n`TopologySpreadConstraint` helps schedule pods with desired skew, but it can not solve the issue: schedule desired\nreplicas to a `zone`, `region` or `cluster`, e.g.\n\n```yaml\nzoneA: 6 Pods\nzoneB: 1 Pods\nzoneC: 2 Pods\n```\n\n## Install\n\n### kube-scheduler\n\n*1* Apply crd\n\n```yaml\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n  name: topologyschedulingpolicies.scheduling.sigs.k8s.io\nspec:\n  conversion:\n    strategy: None\n  group: scheduling.sigs.k8s.io\n  names:\n    kind: TopologySchedulingPolicy\n    listKind: TopologySchedulingPolicyList\n    plural: topologyschedulingpolicies\n    shortNames:\n      - tsp\n      - tsps\n    singular: topologyschedulingpolicy\n  scope: Namespaced\n  version: v1alpha1\n  versions:\n    - name: v1alpha1\n      served: true\n      storage: true\n```\n\n\u003e if your cluster only support kubescheduler.config.k8s.io/v1, please replace this with v1.\n\n*2* deploy scheduler\n\nReplace the kube-scheudler with this one, and add a config like this when starting scheduler.\n\n```yaml\napiVersion: kubescheduler.config.k8s.io/v1beta1\nkind: KubeSchedulerConfiguration\nleaderElection:\n  leaderElect: true\nclientConnection:\n  kubeconfig: \"REPLACE_ME_WITH_KUBE_CONFIG_PATH\"\nprofiles:\n  - schedulerName: default-multicluster\n    plugins:\n      preFilter:\n        enabled:\n          - name: TopologyScheduling\n      filter:\n        enabled:\n          - name: TopologyScheduling\n        disabled:\n          - name: \"*\"\n      score:\n        enabled:\n          - name: TopologyScheduling\n        disabled:\n          - name: \"*\"\n      reserve:\n        enabled:\n          - name: TopologyScheduling\n    pluginConfig:\n      - name: TopologyScheduling\n        args:\n          kubeConfigPath: \"REPLACE_ME_WITH_KUBE_CONFIG_PATH\"\n```\n\u003e If you want to enable multi-cluster, enable the MultiClusterScheduling in the config, as follow:\n```\n      filter:\n        enabled:\n          - name: MultiClusterScheduling\n```\n\n*3* deploy descheduler\n\ndescheduler should be deployed as deployment in cluster\n\n```yaml\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: descheduler-policy-configmap\n  namespace: kube-system\ndata:\n  policy.yaml: |\n    apiVersion: \"descheduler/v1alpha1\"\n    kind: \"DeschedulerPolicy\"\n    strategies:\n      RemovePodsViolatingTopologySchedulingPolicy:\n        enabled: true\n---\nkind: ClusterRole\napiVersion: rbac.authorization.k8s.io/v1\nmetadata:\n  name: descheduler-cluster-role\n  namespace: kube-system\nrules:\n- apiGroups: [\"\"]\n  resources: [\"events\"]\n  verbs: [\"create\", \"update\"]\n- apiGroups: [\"\"]\n  resources: [\"nodes\"]\n  verbs: [\"get\", \"watch\", \"list\"]\n- apiGroups: [\"\"]\n  resources: [\"pods\"]\n  verbs: [\"create\", \"get\", \"watch\", \"list\", \"delete\", \"patch\"]\n- apiGroups: [\"\"]\n  resources: [\"pods/eviction\"]\n  verbs: [\"create\"]\n---\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n  name: descheduler-sa\n  namespace: kube-system\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n  name: descheduler-cluster-role-binding\n  namespace: kube-system\nroleRef:\n  apiGroup: rbac.authorization.k8s.io\n  kind: ClusterRole\n  name: descheduler-cluster-role\nsubjects:\n  - name: descheduler-sa\n    kind: ServiceAccount\n    namespace: kube-system\n---\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: descheduler\n  namespace: kube-system\nspec:\n  selector:\n    matchLabels:\n      app: descheduler\n  replicas: 1\n  template:\n    metadata:\n      labels:\n        app: descheduler\n    spec:\n      affinity:\n        nodeAffinity:\n          requiredDuringSchedulingIgnoredDuringExecution:\n            nodeSelectorTerms:\n            - matchExpressions:\n              - key: type\n                operator: NotIn\n                values:\n                - virtual-kubelet\n      tolerations:\n      - effect: NoSchedule\n        key: role\n        value: not-vk\n        operator: Equal\n      priorityClassName: system-cluster-critical\n      containers:\n      - name: descheduler\n        image: ${you image}\n        volumeMounts:\n        - mountPath: /policy-dir\n          name: policy-volume\n        command:\n        - \"/bin/descheduler\"\n        args:\n        - \"--policy-config-file=/policy-dir/policy.yaml\"\n        - \"--v=3\"\n      restartPolicy: \"Always\"\n      serviceAccountName: descheduler-sa\n      volumes:\n      - name: policy-volume\n        configMap:\n          name: descheduler-policy-configmap\n```\n\n## Use Case\n\n### multi zone\n\n```yaml\napiVersion: scheduling.sigs.k8s.io/v1alpha1\nkind: TopologySchedulingPolicy\nmetadata:\n  name: policy-zone\nspec:\n  deployPlacement:\n    - name: sh-1\n      replicas: 6\n    - name: nj-2\n      replicas: 3\n  labelSelector:\n    matchLabels:\n      cluster-test: \"true\"\n  topologyKey: failure-domain.beta.kubernetes.io/zone\n```\n\n```yaml\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  labels:\n    app: env\n  name: test\n  namespace: default\nspec:\n  replicas: 9\n  selector:\n    matchLabels:\n      app: env\n  template:\n    labels:\n      app: env\n      cluster-test: \"true\"\n      topology-scheduling-policy.scheduling.sigs.k8s.io: policy-zone\n    spec:\n      containers:\n        - image: nginx:latest\n          imagePullPolicy: Always\n          name: nginx\n          resources: { }\n```\n\n### multi region\n\n```yaml\napiVersion: scheduling.sigs.k8s.io/v1alpha1\nkind: TopologySchedulingPolicy\nmetadata:\n  name: policy-region\nspec:\n  deployPlacement:\n    - name: nj\n      replicas: 6\n    - name: sh\n      replicas: 3\n  labelSelector:\n    matchLabels:\n      cluster-test: \"true\"\n  topologyKey: failure-domain.beta.kubernetes.io/region\n```\n\n```yaml\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  labels:\n    app: env\n  name: test\n  namespace: default\nspec:\n  replicas: 9\n  selector:\n    matchLabels:\n      app: env\n  template:\n    metadata:\n      creationTimestamp: null\n      labels:\n        app: env\n        cluster-test: \"true\"\n        topology-scheduling-policy.scheduling.sigs.k8s.io: policy-region\n    spec:\n      containers:\n        - image: nginx:latest\n          imagePullPolicy: Always\n          name: nginx\n          resources: { }\n```\n\n### multi cluster\n\nThis project also can be used in multi-cluster scene by deploy the\n[tensile-kube](https://github.com/virtual-kubelet/tensile-kube), with `descheduler` in `tensile-kube` not deployed.\n\nFor example, we add a label `cluster-name..scheduling.sigs.k8s.io: cluster1` to a virtual node.\n\n```yaml\napiVersion: scheduling.sigs.k8s.io/v1alpha1\nkind: TopologySchedulingPolicy\nmetadata:\n  name: policy-cluster\nspec:\n  deployPlacement:\n    - name: cluster1\n      replicas: 6\n    - name: cluster2\n      replicas: 3\n  labelSelector:\n    matchLabels:\n      cluster-test: \"true\"\n  topologyKey: cluster-name..scheduling.sigs.k8s.io\n```\n\n```yaml\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  labels:\n    app: env\n  name: test\n  namespace: default\nspec:\n  replicas: 9\n  selector:\n    matchLabels:\n      app: env\n  template:\n    metadata:\n      creationTimestamp: null\n      labels:\n        app: env\n        cluster-test: \"true\"\n        cluster-name..scheduling.sigs.k8s.io: policy-cluster\n    spec:\n      containers:\n        - image: nginx:latest\n          imagePullPolicy: Always\n          name: nginx\n          resources: { }\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcwdsuzhou%2Fsuper-scheduling","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcwdsuzhou%2Fsuper-scheduling","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcwdsuzhou%2Fsuper-scheduling/lists"}