{"id":20661793,"url":"https://github.com/mirantis/criproxy","last_synced_at":"2025-08-31T05:33:44.298Z","repository":{"id":57556394,"uuid":"109821784","full_name":"Mirantis/criproxy","owner":"Mirantis","description":"Multiple CRIs per node \u0026 running CRI implementations inside pods","archived":false,"fork":false,"pushed_at":"2020-02-26T12:18:46.000Z","size":586,"stargazers_count":42,"open_issues_count":3,"forks_count":13,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-19T18:43:25.125Z","etag":null,"topics":["container-runtime-interface","kubernetes"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mirantis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-07T10:38:34.000Z","updated_at":"2024-06-28T08:21:33.000Z","dependencies_parsed_at":"2022-09-14T12:22:19.298Z","dependency_job_id":null,"html_url":"https://github.com/Mirantis/criproxy","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/Mirantis/criproxy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirantis%2Fcriproxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirantis%2Fcriproxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirantis%2Fcriproxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirantis%2Fcriproxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mirantis","download_url":"https://codeload.github.com/Mirantis/criproxy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirantis%2Fcriproxy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272940975,"owners_count":25019015,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-31T02:00:09.071Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container-runtime-interface","kubernetes"],"created_at":"2024-11-16T19:11:36.116Z","updated_at":"2025-08-31T05:33:44.266Z","avatar_url":"https://github.com/Mirantis.png","language":"Go","readme":"# CRI Proxy [![CircleCI](https://circleci.com/gh/Mirantis/criproxy/tree/master.svg?style=svg)](https://circleci.com/gh/Mirantis/criproxy/tree/master)\n\nCRI Proxy makes it possible to run several CRI implementations on the\nsame node and run CRI implementations inside pods. CRI Proxy is\ncurrently used by [Virtlet](https://github.com/Mirantis/virtlet)\nproject but it can be used by other CRI implementations, too.\n\nIt supports Kubernetes versions 1.9.x through 1.12.x.\n\n## Installation\n\nIn order to use CRI Proxy on a Kubernetes node you need to install it on your\nsystem and then reconfigure existing kubelet setup to make it use CRI Proxy.\n\n### Installation on kubeadm clusters for use with Virtlet\n\nFor kubeadm-deployed Kubernetes node running Ubuntu 16.04, you can go to\n[Releases](https://github.com/Mirantis/criproxy/releases) page of the\nproject, download latest `criproxy_X.Y.Z_amd64.deb` to the node\nand install it with dpkg:\n```bash\ndpkg -i criproxy_X.Y.Z_amd64.deb\n```\n\nIf you're using\n[kubeadm-dind-cluster](https://github.com/Mirantis/kubeadm-dind-cluster)\nto run your test cluster, you can download\n`criproxy-nodeps_X.Y.Z_amd64.deb` to a DIND node using e.g. `docker\nexec -it kube-node-XX ...` and install it there with `dpkg -i`.\n\nThe packages are currently tailored to support Virtlet deployments\nbut later they can be made more universal.\n\n### Installation on a systemd-based system in a general case\n\nIn order to install the CRI Proxy manually, first you need to download\n`criproxy` binary from\n[Releases](https://github.com/Mirantis/criproxy/releases) page of the\nproject and place it under `/usr/local/bin`.\n\nCreate a file named `/etc/systemd/system/criproxy.service` with\nthe following content (you can also use `systemctl --force edit criproxy.service` for it):\n\n```ini\n[Unit]\nDescription=CRI Proxy\n\n[Service]\nExecStart=/usr/local/bin/criproxy -v 3 -logtostderr -connect /var/run/dockershim.sock,virtlet.cloud:/run/virtlet.sock -listen /run/criproxy.sock\nRestart=always\nStartLimitInterval=0\nRestartSec=10\n\n[Install]\nWantedBy=kubelet.service\n```\n\nYou can remove `-v 3` option to reduce verbosity level of the proxy.\n\n## Reconfiguring kubelet to use CRI Proxy\n\n### Adding dockershim service\n\nNext step is to configure dockershim service. Dockershim can be\nrun using the same binary as kubelet with one catch: Kubernetes 1.8\nprior to 1.8.4 doesn't support `kubelet --experimental-dockershim` in\n`hyperkube`. If that's your case, you can either upgrade Kubernetes\nbinaries on your node or download `kubelet` binary for your version from\n`https://storage.googleapis.com/kubernetes-release/release/vX.Y.Z/bin/linux/amd64/kubelet`\n(put your version instead of vX.Y.Z) and use it for dockershim.\n\nThe idea is to start kubelet with extra flags\n`--experimental-dockershim --port 11250`, other flags being the same\nas those used for kubelet except for `--container-runtime`,\n`--container-runtime-endpoint`, `--image-service-endpoint` and\n`--port`.  More precisely, you need to pass the following flags\n(others, besides container runtime and port options mentioned above,\nare ignored):\n* `--network-plugin`\n* `--hairpin-mode`\n* `--non-masquerade-cidr`\n* `--cni-conf-dir`\n* `--cni-bin-dir`\n* `--docker-endpoint`\n* `--runtime-request-timeout`\n* `--image-pull-progress-deadline`\n* `--streaming-connection-idle-timeout`\n* `--docker-exec-handler`\n* `--seccomp-profile-root`\n* `--pod-infra-container-image`\n* `--runtime-cgroups`\n* `--cgroup-driver`\n* `--network-plugin-mtu`\n* `--address`\n\nCreate a file named `/etc/systemd/system/dockershim.service` with the\nfollowing content, replacing `......` with kubelet command line\narguments (a naive way to get them is just to do `ps aux|grep kubelet`\nif you have `kubelet` service running):\n\n```ini\n[Unit]\nDescription=dockershim for criproxy\n\n[Service]\nExecStart=/usr/bin/kubelet --experimental-dockershim --port 11250 ......\nRestart=always\nStartLimitInterval=0\nRestartSec=10\n\n[Install]\nRequiredBy=criproxy.service\n```\n\n`--port 11250` specifies streaming port to use (it's used for things\nlike `kubectl attach`). If you use another port, you'll also need to\nset `-streamPort XX` option for `criproxy`. If you get errors when\ntrying to do `kubectl exec`, `kubectl attach` or `kubectl port-forward`\non the node with CRI Proxy, this means that CRI Proxy fails to\ndetermine node address properly and you need to pass `-streamUrl`\noption to `criproxy`, e.g. `-streamUrl http://node-ip-address:11250/`.\nThis commonly happens when `--address` flag is passed to kubelet.\nNote that `-streamPort` is ignored if `-streamUrl` is set.\n\nThen enable and start the units after stopping kubelet:\n```bash\nsystemctl stop kubelet\nsystemctl daemon-reload\nsystemctl enable criproxy dockershim\nsystemctl start criproxy dockershim\n```\n\n### Configuring kubelet to use criproxy\n\nYou need to pass the following extra flags to kubelet to make it use CRI Proxy:\n```bash\n--container-runtime=remote \\\n--container-runtime-endpoint=unix:///run/criproxy.sock \\\n--image-service-endpoint=unix:///run/criproxy.sock \\\n--enable-controller-attach-detach=false\n```\nThey should be set in `kubelet.service` file which can be usually found in\n`/lib/systemd/system` directory.\nFinally you will need to do another `systemctl daemon-reload` followed\nby `systemctl start kubelet`.\n\n## How CRI Proxy works\n\nBelow is a diagram depicting the way CRI Proxy works. The basic idea\nis forwarding the requests to different runtimes based on prefixes of\nimage name / pod id / container id prefixes.\n\n![CRI Request Path](criproxy.png)\n\nLet's say CRI Proxy is started as follows:\n```\n/usr/bin/criproxy -v 3 -logtostderr -connect /var/run/dockershim.sock,virtlet.cloud:/run/virtlet.sock -listen /run/criproxy.sock\n```\n\n`-v` option of `criproxy` controls the verbosity here. 0-1 means some\nvery basic logging during startup and displaying serious errors, 2 is\nthe same as 1 plus logging of CRI request errors and 3 causes dumping\nof actual CRI requests and responses except for noisy `List*` and pod\ncontainer/status requests in addition to what's logged on level 2, so\non level 3 most of the output consists mostly of requests that change\nthe state of the runtime. Level 4 enables dumping of pod / container\nstatus requests. Level 5 adds dumping `List*` requests which may cause\nthe log to grow fast. See\n[fixing log throttling](#fixing-log-throttling) below if you're\nstarting CRI Proxy using systemd with log level set to 3 or higher.\n\n`-logtostderr` directs logging output to stderr (it's part of glog configuration)\n\n`-connect /var/run/dockershim.sock,virtlet.cloud:/run/virtlet.sock` specifies the list of\nruntimes that the proxy passes requests to.\n\n`/var/run/dockershim.sock` is a primary runtime that will handle\nunprefixed images and pods without `kubernetes.io/target-runtime`\nannotation.\n\n`virtlet.cloud:/run/virtlet.sock` denotes an alternative runtime\nsocket. This means that image service requests that include image\nnames starting with `virtlet.cloud/` must be directed to the CRI\nimplementation listening on a Unix domain socket at\n`/run/virtlet.sock`. Pods that need to run on `virtlet.cloud` runtime must\nhave `virtlet.cloud` as the value of `kubernetes.io/target-runtime`\nannotation.\n\nThere can be any number of runtimes, although probably using more than\na couple of runtimes is a rare use case.\n\nHere's an example of a pod that needs to run on `virtlet.cloud` runtime:\n```\napiVersion: v1\nkind: Pod\nmetadata:\n  name: cirros-vm\n  annotations:\n    kubernetes.io/target-runtime: virtlet.cloud\n    affinity:\n      nodeAffinity:\n        requiredDuringSchedulingIgnoredDuringExecution:\n          nodeSelectorTerms:\n          - matchExpressions:\n            - key: extraRuntime\n              operator: In\n              values:\n              - virtlet\nspec:\n  containers:\n    - name: cirros-vm\n      image: virtlet.cloud/image-service/cirros\n```\n\nFirst of all, there's `kubernetes.io/target-runtime: virtlet.cloud`\nannotation that directs `RunPodSandbox` requests to `virtlet.cloud` runtime.\n\nThere's also `nodeAffinity` spec that makes the pod run only on the\nnodes that have `extraRuntime=virtlet` label. This is not required\nby CRI Proxy mechanism itself and is related to deployment mechanism\nbeing used.\n\nAnother important part is `virtlet.cloud/image-service/cirros` image name.\nIt means that the image is handled by `virtlet.cloud` runtime and actual\nimage name passed to the runtime is `image-service/cirros`. In case of\nvirtlet this means downloading QCOW2 image from\n`http://image-service/cirros`.\n\nIn order to distinguish between runtimes during requests that don't\ninclude image name or pod annotations such as `RemovePodSandbox`, CRI\nProxy adds prefixes to pod and container ids returned by the runtimes.\n\n## \u003ca name=\"fixing-log-throttling\"\u003e\u003c/a\u003eFixing log throttling\n\nIf you're using log level 3 or higher, journald may throttle CRI Proxy\nlogs. If this is the case, you'll see some delays when checking CRI\nProxy logs with commands like `journalctl -xef -u criproxy`. In order\nto work around this issue, you need to disable journald's rate\nlimiter. To do so, add the following line to\n`/etc/systemd/journald.conf` (if it already has `RateLimitInterval=...`\nline, you just need to change the value to 0)\n\n```\nRateLimitInterval=0\n```\n\nAnd then restart journald:\n```\nsystemctl restart systemd-journald\n```\n\nSee\n[this post](https://www.rootusers.com/how-to-change-log-rate-limiting-in-linux/)\nfor more info on the issue.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirantis%2Fcriproxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmirantis%2Fcriproxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirantis%2Fcriproxy/lists"}