{"id":13643703,"url":"https://github.com/qawolf/crik","last_synced_at":"2025-05-07T11:47:07.897Z","repository":{"id":228362221,"uuid":"767562706","full_name":"qawolf/crik","owner":"qawolf","description":"Checkpoint and Restore in Kubernetes","archived":false,"fork":false,"pushed_at":"2024-05-15T09:55:42.000Z","size":44,"stargazers_count":137,"open_issues_count":2,"forks_count":4,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-05-04T13:17:18.909Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qawolf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-05T14:17:57.000Z","updated_at":"2025-04-08T10:09:14.000Z","dependencies_parsed_at":"2024-03-18T12:43:22.081Z","dependency_job_id":"fa89ebf1-fd1a-4cca-b0ae-b273b0d1c4bf","html_url":"https://github.com/qawolf/crik","commit_stats":null,"previous_names":["qawolf/crik"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qawolf%2Fcrik","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qawolf%2Fcrik/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qawolf%2Fcrik/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qawolf%2Fcrik/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qawolf","download_url":"https://codeload.github.com/qawolf/crik/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252873938,"owners_count":21817708,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:51.483Z","updated_at":"2025-05-07T11:47:07.872Z","avatar_url":"https://github.com/qawolf.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# Checkpoint and Restore in Kubernetes - crik\n\n`crik` is a project that aims to provide checkpoint and restore functionality for Kubernetes pods mainly targeted for\nnode shutdown and restart scenarios. It is a command wrapper that, under the hood, utilizes\n[`criu`](https://github.com/checkpoint-restore/criu) to checkpoint and restore process trees in a `Pod`.\n\n\u003e `crik` is first revealed at KubeCon EU 2024:\n\u003e [The Party Must Go on - Resume Pods After Spot Instance Shut Down - Muvaffak Onuş, QA Wolf](https://kccnceu2024.sched.com/event/1YeP3)\n\nIt is a work in progress and is not ready for production use.\n\n`crik` has two components:\n\n- `crik` - a command wrapper that executes given command and checkpoints it when SIGTERM is received and restores from\n  checkpoint when image directory contains a checkpoint.\n- `manager` - a kubernetes controller that watches `Node` objects and updates its internal map of states so that `crik`\n  can check whether it should checkpoint or restore depending on its node's state.\n\n## Quick Start\n\nThe only pre-requisite is to have a Kubernetes cluster running. You can use `kind` to create a local cluster.\n\n```bash\nkind create cluster\n```\n\nThen, you can deploy the simple-loop example where a counter increases every second and you can delete the pod and see\nthat it continues from where it left off in the new pod.\n\n```bash\nkubectl apply -f examples/simple-loop.yaml\n```\n\nWatch logs:\n\n```bash\nkubectl logs -f simple-loop-0\n```\n\nIn another terminal, delete the pod:\n\n```bash\nkubectl delete pod simple-loop-0\n```\n\nNow, a new pod is created. See that it continues from where it left off:\n\n```bash\nkubectl logs -f simple-loop-0\n```\n\n## Usage\n\nThe application you want to checkpoint and restore should be run with `crik` command, like the following:\n\n```bash\ncrik run -- app-binary\n```\n\nThe following is an example `Dockerfile` for your application that installs `crik` and runs your application. It assumes\nyour application is `entrypoint.sh`.\n\n```Dockerfile\nFROM ubuntu:22.04\n\nRUN apt-get update \u0026\u0026 apt-get install --no-install-recommends --yes gnupg curl ca-certificates\n\n# crik requires criu to be available.\nRUN curl \"https://keyserver.ubuntu.com/pks/lookup?op=get\u0026search=0x4E2A48715C45AEEC077B48169B29EEC9246B6CE2\" | gpg --dearmor \u003e /usr/share/keyrings/criu-ppa.gpg \\\n    \u0026\u0026 echo \"deb [signed-by=/usr/share/keyrings/criu-ppa.gpg] https://ppa.launchpadcontent.net/criu/ppa/ubuntu jammy main\" \u003e /etc/apt/sources.list.d/criu.list \\\n    \u0026\u0026 apt-get update \\\n    \u0026\u0026 apt-get install --no-install-recommends --yes criu iptables\n\n# Install crik\nCOPY --from=ghcr.io/qawolf/crik/crik:v0.1.2 /usr/local/bin/crik /usr/local/bin/crik\n\n# Copy your application\nCOPY entrypoint.sh /entrypoint.sh\n\n# Run your application with crik\nENTRYPOINT [\"crik\", \"run\", \"--\", \"/entrypoint.sh\"]\n```\n\n### Configuration\n\nNot all apps can be checkpointed and restored and for many of them, `criu` may need additional configurations. `crik`\nprovides a high level configuration interface that you can use to configure `crik` for your application. The following\nis the minimum configuration you need to provide for your application and by default `crik` looks for `config.yaml` in\n`/etc/crik` directory.\n\n```yaml\nkind: ConfigMap\nmetadata:\n  name: crik-simple-loop\ndata:\n  config.yaml: |-\n    imageDir: /etc/checkpoint\n```\n\nConfiguration options:\n\n- `imageDir` - the directory where `crik` will store the checkpoint images. It needs to be available in the same path\n  in the new `Pod` as well.\n- `additionalPaths` - additional paths that `crik` will include in the checkpoint and copy back in the new `Pod`. Populate\n  this list if you get `file not found` errors in the restore logs. The paths are relative to root `/` and can be\n  directories or files.\n- `inotifyIncompatiblePaths` - paths that `crik` will delete before taking the checkpoint. Populate this list if you get\n  `fsnotify: \tHandle 0x278:0x2ffb5b cannot be opened` errors in the restore logs. You need to find the inode of the\n  file by converting `0x2ffb5b` to an integer, and then find the path of the file by running `find / -inum \u003cinode\u003e` and\n  add the path to this list. See [this comment](https://github.com/checkpoint-restore/criu/issues/1187#issuecomment-1975557296) for more details.\n\n### Node State Server\n\n\u003e Alpha feature. Not ready for production use.\n\nYou can optionally configure `crik` to take checkpoint only if the node it's running on is going to be shut down. This is\nachieved by deploying a Kubernetes controller that watches `Node` events and updates its internal map of states so that\n`crik` can check whether it should checkpoint or restore depending on its node's state. This may include direct calls\nto the cloud provider's API to check the node's state in the future.\n\nDeploy the controller:\n\n```bash\nhelm upgrade --install node-state-server oci://ghcr.io/qawolf/crik/charts/node-state-server --version 0.1.2\n```\n\nMake sure to include the URL of the server in `crik`'s configuration mounted to your `Pod`.\n\n```yaml\n# Assuming the chart is deployed to default namespace.\nkind: ConfigMap\nmetadata:\n  name: crik-simple-loop\ndata:\n  config.yaml: |-\n    imageDir: /etc/checkpoint\n    nodeStateServerURL: http://crik-node-state-server.default.svc.cluster.local:9376\n```\n\n`crik` will hit the `/node-state` endpoint of the server to get the state of the node it's running on when it receives\nSIGTERM and take checkpoint only if it returns `shutting-down` as the node's state. However, it needs to provide the\nnode name to the server so make sure to add the following environment variable to your container spec in your `Pod`:\n\n```yaml\nenv:\n  - name: KUBERNETES_NODE_NAME\n    valueFrom:\n      fieldRef:\n        fieldPath: spec.nodeName\n```\n\n## Developing\n\nBuild `crik`:\n\n```bash\ngo build -o crik cmd/crik/main.go\n```\n\n## Why not upstream?\n\nTaking checkpoints of processes and restoring them from within the container requires quite a few privileges to be given\nto the container. The best approach is to execute these operations at the container runtime level and today, container\nengines such as CRI-O and Podman do have native support for using `criu` to checkpoint and restore the whole containers\nand there is an ongoing effort to bring this functionality to Kubernetes as well. The first use case being the forensic\nanalysis via checkpoints as described [here](https://kubernetes.io/blog/2023/03/10/forensic-container-analysis/).\n\nWhile it is the better approach, since it's such a low-level change, it's expected to take a while to be available in\nmainstream Kubernetes in an easily consumable way. For example, while taking a checkpoint is possible through `kubelet`\nAPI if you're using CRI-O, restoring it as another `Pod` in a different `Node` is not natively supported yet.\n\n`crik` allows you to use `criu` to checkpoint and restore a `Pod` to another `Node` today without waiting for the native\nsupport in Kubernetes. Once the native support is available, `crik` will utilize it under the hood.\n\n## License\n\nThis project is licensed under the Apache License, Version 2.0 - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqawolf%2Fcrik","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqawolf%2Fcrik","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqawolf%2Fcrik/lists"}