{"id":39926455,"url":"https://github.com/spectrocloud/canvos","last_synced_at":"2026-05-12T20:11:17.937Z","repository":{"id":158929088,"uuid":"629016559","full_name":"spectrocloud/CanvOS","owner":"spectrocloud","description":"A utility for creating Edge artifacts for deploying Palette Edge clusters.","archived":false,"fork":false,"pushed_at":"2026-01-10T21:11:01.000Z","size":2553,"stargazers_count":17,"open_issues_count":30,"forks_count":4,"subscribers_count":6,"default_branch":"main","last_synced_at":"2026-01-11T06:30:45.823Z","etag":null,"topics":["edge","kubernetes","palette"],"latest_commit_sha":null,"homepage":"https://docs.spectrocloud.com/clusters/edge/edgeforge-workflow/","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spectrocloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-04-17T12:59:59.000Z","updated_at":"2026-01-10T21:11:02.000Z","dependencies_parsed_at":"2023-12-17T13:25:49.490Z","dependency_job_id":"9a2a8865-2293-4d27-a75a-4652571433d6","html_url":"https://github.com/spectrocloud/CanvOS","commit_stats":null,"previous_names":[],"tags_count":60,"template":false,"template_full_name":null,"purl":"pkg:github/spectrocloud/CanvOS","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spectrocloud%2FCanvOS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spectrocloud%2FCanvOS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spectrocloud%2FCanvOS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spectrocloud%2FCanvOS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spectrocloud","download_url":"https://codeload.github.com/spectrocloud/CanvOS/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spectrocloud%2FCanvOS/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28547004,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T14:59:57.589Z","status":"ssl_error","status_checked_at":"2026-01-18T14:59:46.540Z","response_time":98,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["edge","kubernetes","palette"],"created_at":"2026-01-18T18:01:05.330Z","updated_at":"2026-05-12T20:11:17.930Z","avatar_url":"https://github.com/spectrocloud.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CanvOS\n\nCanvOS is designed to leverage the Spectro Cloud Edge Forge architecture to build edge artifacts. These artifacts can then be used by Palette for building edge clusters with little to no touch by end users.\n\nWith CanvOS, we leverage Earthly to build all of the artifacts required for edge deployments. From the installer iso to the Kubernetes Provider images, CanvOS makes it simple for you to build the images customized to your needs.\n\nThe base image definitions reside in the Earthfile located in this repo. This defines all of the elements that are required for building the artifacts that can be used by Palette for edge deployments. If customized packages need to be added, simply add the reference to the Dockerfile as you would for any Docker image. When the build command is run, the Earthfile will merge those custom packages into the final image. For a quickstart tutorial see the Knowledgebase section of the Spectro Cloud Docs. There you will find a quickstart tutorial for building your first CanvOS artifacts.\n\n\u003ch1 align=\"center\"\u003e\n \u003cbr\u003e\n \u003cimg alt=\"Edge Components\" src=\"https://raw.githubusercontent.com/spectrocloud/CanvOS/main/images/edge_components.png\"\u003e\n \u003cbr\u003e\n\u003cbr\u003e\n\n## Image Build Architecture\n\n### Base Image\n\nFrom the Kairos project, this is derived from the operating system distribution chosen (currently Ubuntu and OpenSuse-Leap supported). It is pulled down as the base image and some adjustments are made to better support Palette. Those adjustments are used to clean and update the image as well as install some required packages.\n\n### Provider Image\n\nFrom the Base Image, the provider image is used to package in the Kubernetes distribution and version(s) that are part of the build. This layer is required to initialize the system and prepare it for configuration to build the Kubernetes cluster.\n\n### Installer Image\n\nFrom the base image, this image is used to provide the initial flashing of a device (bare-metal or virtual machine). This image contains the user-data configuration that has been provided in `user-data`. It will also contain the contents of any content bundle for pre-staged builds. Pre-staged builds can be used to embed all of the artifacts that are required to build a cluster. These artifacts include Helm charts, manifests, and container images. These images are loaded into containerd when the cluster is initialized eliminating the need for the initial download. For more information on how to build pre-loaded content checkout the Palette Docs at [Build your Own Content](https://docs.spectrocloud.com/clusters/edge/edgeforge-workflow/build-content-bundle).\n\n### Custom Configuration\n\nFor advanced use cases, there may be a need to add additional packages not included in the [Base Images](https://github.com/kairos-io/kairos/tree/master/images). If those packages or configuration elements need to be added, they can be included in the empty `Dockerfile` located in this repo and they will be included in the build process and output artifacts.\n\n### Custom Hardware Specs Lookup\n\nA custom hardware specs lookup file can be provided to extend or override the default GPU specifications. This is useful for adding custom hardware entries or overriding default specifications.\n\n#### Location\n\nPlace the custom lookup file at:\n```\noverlay/files/etc/spectrocloud/custom-hardware-specs-lookup.json\n```\n\nThis file will be copied to `/etc/spectrocloud/custom-hardware-specs-lookup.json` in the final image.\n\n#### Configuration\n\nThe file must be a valid JSON array. These entries will be merged with the default hardware specs lookup table, with the custom entries taking precedence for matching `vendorID` and `deviceID` combinations.\n\n#### Sample Entry\n\nHere's an example entry structure:\n\n```json\n[\n {\n \"marketingName\": \"NVIDIA Quadro RTX 6000\",\n \"vRAM\": 24,\n \"migCapable\": false,\n \"vendor\": \"NVIDIA\",\n \"vendorID\": \"10de\",\n \"deviceID\": [\n \"1e36\",\n \"1e78\"\n ],\n \"architecture\": \"Turing\"\n }\n]\n```\n\n#### Field Descriptions\n\n| Field | Description | Example |\n|-------|-------------|---------|\n| `marketingName` | Display name for the hardware | `\"NVIDIA Quadro RTX 6000\"` |\n| `vRAM` | Video RAM in GB (decimal) | `24` |\n| `migCapable` | Whether Multi-Instance GPU is supported (for NVIDIA GPUs) | `false` |\n| `vendor` | Vendor name | `\"NVIDIA\"`, `\"AMD\"`, `\"Intel\"` |\n| `vendorID` | PCI vendor ID in hexadecimal (lowercase) | `\"10de\"` |\n| `deviceID` | Array of PCI device IDs in hexadecimal (can have multiple IDs for same model) | `[\"1e36\", \"1e78\"]` |\n| `architecture` | GPU architecture name | `\"Turing\"`, `\"Ampere\"`, `\"Hopper\"`, `\"Blackwell\"` |\n\n#### Usage\n\nThere are two ways to provide the custom hardware specs lookup file:\n\n##### Method 1: During CanvOS Build (Recommended for Pre-staged Images)\n\n1. Create the file at `overlay/files/etc/spectrocloud/custom-hardware-specs-lookup.json`\n2. Add custom entries as a JSON array\n3. Build the images - the custom lookup file will be included automatically\n4. Custom entries will override default entries for matching `vendorID`+`deviceID` combinations\n\n##### Method 2: Via user-data (Runtime Configuration)\n\nThe custom hardware specs lookup file can be provided via user-data using the `boot.after` stage (recommended) or `boot` stage. Note that `/etc` is read-only at runtime, but cloud-init can write files during boot stages.\n\nAdd the following to the `user-data` file (replace the example entry with actual hardware specs):\n\n```yaml\n#cloud-config\nstages:\n boot.after:\n - files:\n - path: \"/etc/spectrocloud/custom-hardware-specs-lookup.json\"\n permissions: 0644\n owner: 0\n group: 0\n content: |\n [\n {\n \"marketingName\": \"NVIDIA Quadro RTX 6000\",\n \"vRAM\": 24,\n \"migCapable\": false,\n \"vendor\": \"NVIDIA\",\n \"vendorID\": \"10de\",\n \"deviceID\": [\n \"1e36\",\n \"1e78\"\n ],\n \"architecture\": \"Turing\"\n }\n ]\n name: \"Configure custom hardware specs lookup\"\n```\n\n### Basic Usage\n\n1. Clone the repo at [CanvOS](https://github.com/spectrocloud/CanvOS.git)\n\nNote: If you are building the images behind a proxy server, you may need to configure your git to let it use your proxy server.\n\n```\ngit config --global http.proxy \u003cyour-proxy-server\u003e\ngit config --global https.proxy \u003cyour-proxy-server\u003e\ngit config --global http.sslCAinfo \u003cyour-cert-path\u003e\ngit config --global https.sslCAinfo \u003cyour-cert-path\u003e\n# git config --global http.sslVerify False\n# git config --global https.sslVerify False\n```\n\n```shell\ngit clone https://github.com/spectrocloud/CanvOS.git\n```\n\n**Sample Output**\n\n```shell\nCloning into 'CanvOS'...\nremote: Enumerating objects: 133, done.\nremote: Counting objects: 100% (133/133), done.\nremote: Compressing objects: 100% (88/88), done.\nReceiving objects: 100% (133/133), 40.16 KiB | 5.02 MiB/s, done.\nResolving deltas: 100% (60/60), done.\nremote: Total 133 (delta 60), reused 101 (delta 32), pack-reused 0\n```\n\n2. Change into the `CanvOS` directory that was created.\n\n```shell\ncd CanvOS\n```\n\n3. View Available tags\n\n```shell\ngit tag\n\nv3.3.3\nv3.4.0\nv3.4.1\nv3.4.3\n\nv4.1.0\nv4.2.3\n\nv4.5.11\n```\n\n4. Checkout the desired tag\n\n\u003e **Note:** If you have a dedicated or on-prem instance of Palette, identify the correct agent version and checkout the corresponding tag. For example, if your agent version is v4.6.16, checkout tag v4.6.16.\n\u003e\n\u003e ```shell\n\u003e curl --location --request GET 'https://\u003cpalette-endpoint\u003e/v1/services/stylus/version' --header 'Content-Type: application/json' --header 'Apikey: \u003capi-key\u003e' | jq --raw-output '.spec.latestVersion.content | match(\"version: ([^\\n]+)\").captures[0].string'\n\u003e # Sample Output\n\u003e v4.6.16\n\u003e\n\u003e ```\n\n```shell\ngit checkout \u003ctag version\u003e\n```\n\n**Sample Output**\n\n```shell\ngit checkout v4.2.3\nNote: switching to 'v4.2.3'.\n\nYou are in 'detached HEAD' state. You can look around, make experimental\nchanges and commit them, and you can discard any commits you make in this\nstate without impacting any branches by switching back to a branch.\n\nIf you want to create a new branch to retain commits you create, you may\ndo so (now or later) by using -c with the switch command. Example:\n```\n\n1. Copy the .arg.template file to .arg\n\n```shell\ncp .arg.template .arg\n```\n\n6. To build RHEL core, RHEL FIPS, RHEL 9 STIG, or Ubuntu fips, sles base images switch to respective directories and build the base image.\n The base image built can be passed as argument to build the installer and provider images.\n Follow the instructions in the respective sub-folders (rhel-fips, rhel-stig, ubuntu-fips) to create base images.\n For ubuntu-fips, this image can be used as base image - `gcr.io/spectro-images-public/ubuntu-fips:v3.0.11`\n For RHEL 9 STIG, see `rhel-stig/README.md` for build instructions.\n Skip this step if your base image is ubuntu or opensuse-leap. If you are building ubuntu or opensuse-leap installer images, do not pass the BASE_IMAGE attribute as an arg to build command.\n\n7. Modify the `.arg` file as needed. Primarily, you must define the tag you want to use for your images. For example, if the operating system is `ubuntu` and the tag is `demo`, the image artefact will name as `ttl.sh/ubuntu:k3s-1.25.2-v3.4.3-demo`. The **.arg** file defines the following variables:\n\n| Parameter | Description | Type | Default Value |\n| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -------------------------- |\n| CUSTOM_TAG | Environment name for provider image tagging. The default value is `demo`. | String | `demo` |\n| IMAGE_REGISTRY | Image registry name that will store the image artifacts. The default value points to the _ttl.sh_ image registry, an anonymous and ephemeral Docker image registry where images live for a maximum of 24 hours by default. If you wish to make the images exist longer than 24 hours, you can use any other image registry to suit your needs. | String | `ttl.sh` |\n| OS_DISTRIBUTION | OS distribution of your choice. For example, it can be `ubuntu`, `opensuse-leap`, `rhel` or `sles` | String | `ubuntu` |\n| IMAGE_REPO | Image repository name in your chosen registry. | String | `$OS_DISTRIBUTION` |\n| OS_VERSION | OS version. For Ubuntu, the possible values are `20`, and `22`. Whereas for openSUSE Leap, the possible value is `15.6`. For sles, possible values are `5.4`. This example uses `22` for Ubuntu. | String | `22` |\n| K8S_DISTRIBUTION | Kubernetes distribution name. It can be one of these: `k3s`, `rke2`, `kubeadm`, `kubeadm-fips`, or `nodeadm`. | String | `k3s` |\n| ISO_NAME | Name of the Edge installer ISO image. In this example, the name is _palette-edge-installer_. | String | `palette-edge-installer` |\n| ARCH | Type of platform to use for the build. Used for Cross Platform Build (arm64 to amd64 as example). | string | `amd64` |\n| BASE_IMAGE | Base image to be used for building installer and provider images. | String | |\n| FIPS_ENABLED | to generate FIPS compliant binaries. `true` or `false` | string | `false` |\n| HTTP_PROXY | URL of the HTTP Proxy server to be used if needed (Optional) | string | |\n| HTTPS_PROXY | URL of the HTTPS Proxy server to be used if needed (Optional) | string | |\n| NO_PROXY | URLS that should be excluded from proxying (Optional) | string | |\n| UPDATE_KERNEL | Determines whether to upgrade the Kernel version to the latest from the upstream OS provider | boolean | `false` |\n| DISABLE_SELINUX | Disable selinux in the operating system. Some applications (like Kubevirt) do not like selinux | boolean | `true` |\n| CIS_HARDENING | Enable CIS Benchmark hardening for the image. When set to `true`, applies CIS Benchmark security controls during the build. | boolean | `false` |\n| CLUSTERCONFIG | Path of the cluster config | string | |\n| IS_UKI | Build UKI(Trusted boot) images | boolean | `false` |\n| UKI_BRING_YOUR_OWN_KEYS | Bring your own public/private key pairs if this is set to true. Otherwise, CanvOS will generate the key pair. | boolean | `false` |\n| INCLUDE_MS_SECUREBOOT_KEYS | Include Microsoft 3rd Party UEFI CA certificate in generated keys | boolean | `true` |\n| AUTO_ENROLL_SECUREBOOT_KEYS | Auto enroll SecureBoot keys when device boots up and is in setup mode of secure boot | boolean | `true` |\n| EDGE_CUSTOM_CONFIG | Path to edge custom configuration file | string | `.edge-custom-config.yaml` |\n| MAAS_IMAGE_NAME | Custom name for the final MAAS image (without .raw.gz extension). Only used when building MAAS images. | string | `kairos-ubuntu-maas` |\n| IS_MAAS | Build MAAS-compatible disk images. Set to `true` when building for MAAS deployment. | boolean | `false` |\n\n\u003eFor use cases where UEFI installers are burned into a USB and then edge hosts are booted with the USB. Use the OSBUILDER_VERSION=0.300.4 in the .arg file with canvos when building the installer.\nVersion 0.300.3 does not support UEFI builds loading from USB.\n\n\u003e If there are certs to be added, place them in the `certs` folder.\n\n1. (Optional) If you are building the images behind a proxy server, you may need to modify your docker daemon settings to let it use your proxy server. You can refer this [tutorial](https://docs.docker.com/config/daemon/systemd/#httphttps-proxy).\n\n2. Build the images with the following command. Use the `system.uri` output when creating the cluster profile for the Edge host.\n\n```shell\n./earthly.sh +build-all-images --ARCH=amd64\n```\n\nTo build FIPS complaint images or ARM images, specify the BASE_IMAGE and ARCH in the .arg file or as command line arguments.\n`k3s` does not FIPS and rke2 is by default `FIPS` compliant.\n\nTo build just the installer image\n\n```shell\n./earthly.sh +iso --ARCH=amd64\n```\n\n### Building AWS Cloud Images\n\nCanvOS can build Amazon Machine Images (AMIs) for AWS EC2 deployment. The build process converts the raw disk image into an AMI that can be launched directly in AWS.\n\n#### Prerequisites\n\n1. **AWS Credentials**: Configure AWS credentials in the `.secret` file in the repo directory.\n\n **Option 1: Using AWS Profile (Recommended)**\n ```shell\n AWS_PROFILE=\"production\"\n ```\n\n **Option 2: Using Access Keys**\n ```shell\n AWS_PROFILE=\"\"\n AWS_ACCESS_KEY_ID=\"ACCESS_EXAMPLE_KEY\"\n AWS_SECRET_ACCESS_KEY=\"SECRET_EXAMPLE_KEY\"\n ```\n\n2. **AWS Configuration**: Add the following to your `.arg` file:\n ```shell\n # AWS Cloud Image Configuration\n REGION=\"us-east-1\"\n S3_BUCKET=\"my-canvos-images\"\n S3_KEY=\"\" # Optional: defaults to raw file name if not set\n ```\n\n3. **User Data Configuration**: Create or update `user-data` in the CanvOS root directory with your cloud-init configuration:\n ```yaml\n #cloud-config\n install:\n device: /dev/sda\n reboot: true\n poweroff: false\n stylus:\n debug: true\n site:\n paletteEndpoint: xxxx.spectrocloud.com\n autoRegister: true\n edgeHostToken: xxxxxxxxxxx\n insecureSkipVerify: false\n ```\n\n#### Building the AWS Cloud Image\n\nRun the following command to build the AWS cloud image:\n\n```shell\nearthly -P +aws-cloud-image --ARCH=amd64\n```\n\nThe `-P` flag enables privileged mode (required for Docker-in-Docker operations).\n\n#### Build Process\n\nThe AWS cloud image build consists of two steps:\n\n1. **Step 1**: Builds the cloud image (raw disk image) using auroraboot\n - Creates a raw disk image from the installer image\n - Includes user-data configuration\n - Supports content bundles and cluster configs\n\n2. **Step 2**: Converts raw image to AMI\n - Uploads raw image to S3 bucket\n - Imports snapshot from S3\n - Registers AMI in EC2\n - Returns AMI ID upon completion\n\n#### Output\n\nAfter a successful build, the AMI will be registered in your AWS account. The build output will display:\n- AMI ID (e.g., `ami-0123456789abcdef0`)\n- AMI Name (based on `S3_KEY` or auto-generated)\n- Snapshot ID (created during import)\n\n#### IAM Permissions Required\n\nThe AWS credentials must have the following permissions:\n- `s3:PutObject`, `s3:GetObject`, `s3:ListBucket` on the S3 bucket\n- `ec2:ImportSnapshot`, `ec2:DescribeImportSnapshotTasks`\n- `ec2:DescribeSnapshots`, `ec2:RegisterImage`\n- `ec2:DescribeImages`, `ec2:CreateTags`\n\n#### Troubleshooting\n\n- **\"RAW file not found\"**: Ensure `+cloud-image` target completed successfully\n- **\"Missing required configuration variables\"**: Verify `REGION` and `S3_BUCKET` are set in `.arg` file\n- **\"AWS credentials not found\"**: Check `.secret` file has valid AWS credentials\n- **\"Snapshot import failed\"**: Verify IAM permissions and S3 bucket accessibility\n\nFor more detailed information, see [AWS Cloud Image Design Document](docs/aws-cloud-image-design.md).\n\nTo build the provider images\n\n```shell\n./earthly.sh +build-provider-images --ARCH=amd64\n```\n\nTo build the fips enabled ubuntu installer image\n\n```shell\n./earthly.sh +iso --BASE_IMAGE=gcr.io/spectro-images-public/ubuntu-fips:v3.0.11 --FIPS_ENABLED=true --ARCH=amd64 --PE_VERSION=v4.4.0\n```\n\nTo build the RHEL 9 STIG installer image (non-FIPS)\n\n```shell\n# First, build and push the base image (see rhel-stig/README.md)\n# Then use the full registry path:\n./earthly.sh +iso --BASE_IMAGE=\u003cyour-registry\u003e/rhel9-byoi-stig:latest --OS_DISTRIBUTION=rhel --ARCH=amd64\n```\n\nTo build the RHEL 9 STIG FIPS installer image\n\n```shell\n# First, build and push the base image (see rhel-stig/README.md)\n# Then use the full registry path:\n./earthly.sh +iso --BASE_IMAGE=\u003cyour-registry\u003e/rhel9-byoi-stig-fips:latest --OS_DISTRIBUTION=rhel --FIPS_ENABLED=true --ARCH=amd64\n```\n\n**Note**: For RHEL 9 STIG images, you must:\n1. First build the base image using `rhel-stig/build.sh.rhel9` (see `rhel-stig/README.md`)\n2. Tag and push the image to a Docker registry (Earthly cannot use local-only images)\n3. Use the full registry path in the `BASE_IMAGE` argument\n\nRHEL 9 STIG images require Red Hat subscription credentials and apply DISA STIG security hardening. Firewall rules must be configured at runtime for Palette cluster operations - see `rhel-stig/README.md` for firewall configuration details.\n\nOutput\n\n```shell\n###################################################################################################\n\nPASTE THE CONTENTS BELOW INTO YOUR CLUSTER PROFILE IN PALETTE BELOW THE \"OPTIONS\" ATTRIBUTE\n\n###################################################################################################\n\n\nsystem.uri: \"{{ .spectro.pack.edge-native-byoi.options.system.registry }}/{{ .spectro.pack.edge-native-byoi.options.system.repo }}:{{ .spectro.pack.edge-native-byoi.options.system.k8sDistribution }}-{{ .spectro.system.kubernetes.version }}-{{ .spectro.pack.edge-native-byoi.options.system.peVersion }}-{{ .spectro.pack.edge-native-byoi.options.system.customTag }}\"\n\n\nsystem.registry: ttl.sh\nsystem.repo: ubuntu\nsystem.k8sDistribution: k3s\nsystem.osName: ubuntu\nsystem.peVersion: v4.2.3\nsystem.customTag: demo\nsystem.osVersion: 22\n```\n\n10. Validate the expected artifacts are created, the ISO image and the provider OS images.\n\n```shell\nls build/ \u0026\u0026 docker images\n\npalette-edge-installer.iso\npalette-edge-installer.iso.sha256\n\n# Output\nREPOSITORY TAG IMAGE ID CREATED SIZE\nttl.sh/ubuntu k3s-1.24.6-v4.2.3-demo cad8acdd2797 17 hours ago 4.62GB\nttl.sh/ubuntu k3s-1.24.6-v4.2.3-demo_linux_amd64 cad8acdd2797 17 hours ago 4.62GB\nttl.sh/ubuntu k3s-1.25.2-v4.2.3-demo f6e490f53971 17 hours ago 4.62GB\nttl.sh/ubuntu k3s-1.25.2-v4.2.3-demo_linux_amd64 f6e490f53971 17 hours ago 4.62GB\n```\n\nEarthly is a multi-architecture build tool. In this example we are building images for AMD64 hardware which is reflected by the tags above. In the future we will support ARM64 builds and those tags will be included. We only need to push the image tag that DOES NOT have the architecture reference i.e `linux_amd64` in the above example.\n\n11. The provider images are by default not pushed to a registry. You can push the images by using the `docker push` command and reference the created images.\n\n```shell\ndocker push ttl.sh/ubuntu:k3s-1.25.2-v4.2.3-demo\n```\n\n\u003e ⚠️ The default registry, [ttl.sh](https://ttl.sh/) is a short-lived registry. Images in the ttl.sh registry have a default time to live of\n\u003e 24 hours. Once the time limit is up, the images will automatically be removed. To use a permanent registry, set the `.arg` file's `IMAGE_REGISTRY` parameter with the URL of your image registry.\n\n12. Create a cluster profile using the command output. Use the [Model Edge Cluster Profile](https://docs.spectrocloud.com/clusters/edge/site-deployment/model-profile) to help you complete this step.\n\n13. Flash VM or Baremetal device with the generated ISO. Refer to the [Prepare Edge Host for Installation](https://docs.spectrocloud.com/clusters/edge/site-deployment/stage) guide for additional guidance.\n\n14. Register the Edge host with Palette. Checkout the [Register Edge Host](https://docs.spectrocloud.com/clusters/edge/site-deployment/site-installation/edge-host-registration) guide.\n\n15. Build a cluster in [Palette](https://console.spectrocloud.com).\n\n### How-Tos\n\n- [Building Edge Native Artifacts](\u003c[https://docs.spectrocloud.com/clusters/edge/edgeforge-workflow/palette-canvos](https://deploy-preview-1318--docs-spectrocloud.netlify.app/clusters/edge/edgeforge-workflow/palette-canvos)\u003e)\n\n### Building ARM64 Artifacts for Nvidia Jetson devices\n\n1. Your .arg file should contain these values\n\n```shell\nBASE_IMAGE=quay.io/kairos/ubuntu:20.04-core-arm64-nvidia-jetson-agx-orin-v2.4.3\nARCH=arm64\nplatform=linux/arm64\n```\n\n2. `./earthly.sh +build-all-images`\n\n### Prepare keys for Trusted Boot\n\n1. In .arg, we have these options to configure the key generation for Trusted Boot.\n\n```\nUKI_BRING_YOUR_OWN_KEYS=false # set to true if you want to to use your own public/private key pairs to generate SecureBoot keys\nINCLUDE_MS_SECUREBOOT_KEYS=true # if you want to include Microsoft 3rd Party UEFI CA certificate in generated keys\n```\n\n2. Copy required keys into secure-boot directory. It should look like this:\n\n```shell\nsecure-boot/\n| enrollment\n| exported-keys \u003c-- keys exported from hardware\n| | db\n| | KEK\n| private-keys \u003c-- Will be used if UKI_BRING_YOUR_OWN_KEYS=true, otherwise CanvOS will generate the keys\n| | db.key\n| | KEK.key\n| | PK.key\n| | tpm2-pcr-private.pem\n| public-keys \u003c-- Will be used if UKI_BRING_YOUR_OWN_KEYS=true, otherwise CanvOS will generate the keys\n| | db.pem\n| | KEK.pem\n| | PK.pem\n```\n\n3. Generate keys for Trusted Boot\n\n```shell\n./earthly.sh +uki-genkey --MY_ORG=\"ACME Corp\" --EXPIRATION_IN_DAYS=5475\n```\n\n### Build Trusted Boot Images\n\n1. Your .arg file should contain these values.\n\n```shell\nIS_UKI=true\nAUTO_ENROLL_SECUREBOOT_KEYS=false # Auto enroll SecureBoot keys when device boots up and is in setup mode of secure boot\n```\n\n2. `./earthly.sh +build-all-images`\n\n### Building MAAS Images\n\nMAAS (Metal as a Service) is Canonical's bare-metal provisioning tool. CanvOS can build MAAS-compatible disk images in `dd.gz` format that can be uploaded directly to MAAS for deployment.\n\n#### Prerequisites\n\n- MAAS image builds only support Ubuntu (OS_DISTRIBUTION must be set to `ubuntu`)\n- The build process requires privileged mode (handled automatically by `earthly.sh`)\n\n#### Configuration\n\n1. In your `.arg` file, ensure the following settings:\n\n```shell\nOS_DISTRIBUTION=ubuntu\nOS_VERSION=22 # or 24\nARCH=amd64\n```\n\n2. (Optional) Set a custom name for the MAAS image:\n\n```shell\nMAAS_IMAGE_NAME=my-custom-maas-image\n```\n\nIf not specified, the default name will be `kairos-ubuntu-maas`. The final output will be in `dd.gz` format (e.g., `my-custom-maas-image.raw.gz`).\n\n#### Building the MAAS Image\n\nRun the following command to build the MAAS image:\n\n```shell\n./earthly.sh +maas-image\n```\n\nThe build process consists of two steps:\n1. **Step 1**: Generates the Kairos raw disk image from the installer image\n2. **Step 2**: Creates a composite MAAS image that includes:\n - Kairos OS partitions (EFI, OEM, Recovery)\n - Ubuntu root filesystem for MAAS deployment\n - Curtin hooks for MAAS integration\n - Cloud-init scripts for post-deployment configuration\n\n#### Output\n\nAfter a successful build, you will find the compressed MAAS image in the `build/` directory:\n\n```shell\nls build/\n\nmy-custom-maas-image.raw.gz\nmy-custom-maas-image.raw.gz.sha256\n```\n\nThe output includes:\n- **Compressed disk image** (`.raw.gz`): The MAAS-compatible disk image in `dd.gz` format\n- **SHA256 checksum** (`.sha256`): Checksum file for verification\n\n#### Uploading to MAAS\n\n1. Upload the `.raw.gz` file to MAAS as a custom image:\n - Go to MAAS UI → Images → Custom Images\n - Click \"Upload custom image\"\n - Select the `.raw.gz` file\n - Specify format as `dd.gz`\n - MAAS will automatically decompress the image during upload\n Or\n Use maas command to upload the custom image to MAAS \n Example:\n maas login admin http://\u003cMAAS_URL\u003e:5240/MAAS \u003cAPI_KEY\u003e\n maas admin boot-resources create \\\n name='custom/my-raw-image' \\\n title='My Custom RAW Image' \\\n architecture='amd64/generic' \\\n filetype='ddgz' \\\n content@=/path/to/image.raw.gz\n \n\n2. The image is now ready to be deployed to bare-metal servers via MAAS.\n\n#### Image Contents\n\nThe MAAS image includes:\n- **Content bundles**: Pre-staged container images and artifacts (if provided during build)\n- **Cluster config (SPC)**: Cluster definition files (if `CLUSTERCONFIG` is set in `.arg`)\n- **Local UI**: Local management interface (if `local-ui.tar` is provided)\n- **Curtin hooks**: Custom installation hooks for MAAS deployment\n- **Cloud-init integration**: Automatic userdata processing and recovery mode boot\n\n#### Troubleshooting\n\n- **Build fails with \"OS_DISTRIBUTION is not set\"**: Ensure `OS_DISTRIBUTION=ubuntu` is set in your `.arg` file\n- **Build fails with \"OS_DISTRIBUTION is not ubuntu\"**: MAAS images only support Ubuntu. Change `OS_DISTRIBUTION` to `ubuntu` in your `.arg` file\n- **Image not found after build**: Check that the build completed successfully and look for the `.raw.gz` file in the `build/` directory\n\n### Building with private registry\n\n1. Make sure you have logged into your registry using docker login\n2. In .arg, add following entries\n\n```shell\nSPECTRO_LUET_REPO=reg.xxx.com\nSPECTRO_PUB_REPO=reg.xxx.com\nKAIROS_BASE_IMAGE_URL=reg.xxx.com\n```\n\n3. Make sure you have following images and your base image retagged to your repo\n\n```shell\ngcr.io/spectro-images-public/earthly/earthly:v0.8.5 to reg.xxx.com/earthly/earthly:v0.8.5\ngcr.io/spectro-images-public/earthly/buildkitd:v0.8.5 to reg.xxx.com/earthly/buildkitd:v0.8.5\ngcr.io/spectro-images-public/canvos/alpine-cert:v1.0.0 to reg.xxx.com/canvos/alpine-cert:v1.0.0\ngcr.io/spectro-images-public/osbuilder-tools:v0.7.11 to reg.xxx.com/osbuilder-tools:v0.7.11\ngcr.io/spectro-images-public/stylus-framework-linux-amd64:v4.3.2 to reg.xxx.com/stylus-framework-linux-amd64:v4.3.2\ngcr.io/spectro-images-public/kairos-io/provider-kubeadm:v4.3.1 to reg.xxx.com/kairos-io/provider-kubeadm:v4.3.1\ngcr.io/spectro-images-public/kairos-io/provider-k3s:v4.2.1 to reg.xxx.com/kairos-io/provider-k3s:v4.2.1\ngcr.io/spectro-images-public/kairos-io/provider-rke2:v4.1.1 to reg.xxx.com/kairos-io/provider-rke2:v4.1.1\n```\n\n4. Prepare luet auth config\n\n```shell\ncp spectro-luet-auth.yaml.template spectro-luet-auth.yaml\n# modify serveraddess, username and password in spectro-luet-auth.yaml to yours\n```\n\n5. Build the image using earthly installed on the host\n\n```shell\nearthly --push +build-all-images\n```\n\n### Building Installer Image with public key for verifying signed content\n\n1. Copy the .edge.custom-config.yaml.template file to .edge.custom-config.yaml\n\n```shell\ncp .edge.custom-config.yaml.template .edge.custom-config.yaml\n```\n\n2. Edit the property signing.publicKey in `.edge.custom-config.yaml`\n\n3. Include the following property in `.arg` file\n\n```\n...\n\nEDGE_CUSTOM_CONFIG=/path/to/.edge.custom-config.yaml\n```\n\n4. Build the image using earthly installed on the host\n\n```shell\nearthly --push +build-all-images\n```\n\n### Audit Logs User Customisation\n\n#### Configuration\n\nrsyslog config file: `overlay/files/etc/rsyslog.d/49-stylus.conf` copied to `/etc/rsyslog.d/49-stylus.conf`\nlogrotate config file: `overlay/files/etc/logrotate.d/stylus.conf` copied to `/etc/logrotate.d/stylus.conf`\n\n#### Send stylus audit events to user file\n\nUsers can log stylus audit events to additional files, in addition to `/var/log/stylus-audit.log`. To log stylus audit events to custom files, create a configuration file in the `overlay/files/etc/rsyslog.d` directory named `\u003cfilename\u003e.conf` (must be before `49-stylus.conf` lexicographically).\n\nExample: `48-audit.conf`\n\nUsers can use the following configuration as a base for their filtering logic. replace `\u003clog file name\u003e` with the desired file name\n\n```\n$PrivDropToUser root\n$PrivDropToGroup root\nif ($syslogfacility-text == 'local7' and $syslogseverity-text == 'notice' and $syslogtag contains 'stylus-audit') then {\n action(\n type=\"omfile\"\n file=\"\u003clog file name\u003e\"\n )\n}\n```\n\n#### Send user application audit events to stylus audit file\n\nTo include user application audit events in the `/var/log/stylus-audit.log` file, add the following to the same configuration file (e.g. `48-audit.conf`) or create a new config file before `49-stylus.conf`:\n\n`\u003cuser app name\u003e` : user application name or tag\n\n```\n$PrivDropToUser root\n$PrivDropToGroup root\n$Umask 0000\n$template ForwardFormat,\"\u003c%pri%\u003e1 %timestamp:::date-rfc3339% %HOSTNAME% %syslogtag% %procid% - - %msg%\\n\"\nif ($syslogfacility-text == 'local7' and $syslogseverity-text == 'notice' and $syslogtag contains '\u003cuser app name\u003e') then {\n action(\n type=\"omfile\"\n file=\"/var/log/stylus-audit.log\"\n FileCreateMode=\"0600\"\n fileowner=\"root\"\n template=\"ForwardFormat\"\n )\n}\n```\n\nTo display user audit entries on the Local UI dashboard, audit entries must be logged in RFC 5424 format with the message (`msg`) part in JSON format. This JSON message must include the following keys: `edgeHostId`, `contentMsg`, `action`, `actor`, `actorType`, `resourceId`, `resourceName`, `resourceKind`\n\nExample syslog entry\n\n```\n\u003c189\u003e1 2024-07-23T15:35:32.644461+00:00 edge-ce0a38422e4662887313fb673bbfb2a2 stylus-audit[2911]: 2911 - - {\"edgeHostId\":\"edge-ce0a38422e4662887313fb6 73bbfb2a2\",\"contentMsg\":\"kairos password reset failed\",\"action\":\"activity\",\"actor\":\"kairos\",\"actorType\":\"user\",\"resourceId\":\"kairos\",\"resourceName\":\"kairos\",\"resourceKind\":\"user\"}\n```\n\nEntries without these keys in the MSG part of RFC 5424 will still be logged to the stylus-audit.log file but will not be displayed on LocalUI.\n\n# cgroup v2\n\ncgroup v2 is the next version of the Linux cgroup API. cgroup v2 provides a unified control system with enhanced resource management capabilities.\nKubernetes has deprecated cgroup v1. Kubelet will no longer start on a cgroup v1 node by default from k8s v1.35.x. To disable this setting a cluster admin should set failCgroupV1 to false in the [kubelet configuration](https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/) file or while building the installer/provider images set cgroup v2 as default.\n\nIn the userdata file, while building the installer add systemd.unified_cgroup_hierarchy=1\n\n```\n#cloud-config\ninstall:\n grub_options:\n extra_cmdline: \"systemd.unified_cgroup_hierarchy=1\"\n```\n\nor if you are upgrading from k8s 1.34.x to 1.35.x and the operating system defaults to cgroup v1, build the provider image with cgroupv2 as default by uncommenting the following line in the Dockerfile.\n```\nRUN sed -i 's|\\(set baseCmd=\"[^\"]*\\)\"|\\1 systemd.unified_cgroup_hierarchy=1\"|' \\\n /etc/cos/bootargs.cfg\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspectrocloud%2Fcanvos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspectrocloud%2Fcanvos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspectrocloud%2Fcanvos/lists"}