{"id":13646205,"url":"https://github.com/coreos/mantle","last_synced_at":"2025-04-21T17:32:33.664Z","repository":{"id":24171924,"uuid":"27562362","full_name":"coreos/mantle","owner":"coreos","description":"Mantle: Gluing Container Linux together","archived":true,"fork":false,"pushed_at":"2020-10-12T23:09:53.000Z","size":26502,"stargazers_count":95,"open_issues_count":47,"forks_count":76,"subscribers_count":20,"default_branch":"cl","last_synced_at":"2024-11-09T19:41:35.577Z","etag":null,"topics":["coreos","go"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coreos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"code-of-conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-12-04T22:09:02.000Z","updated_at":"2024-04-09T19:12:53.000Z","dependencies_parsed_at":"2022-08-06T01:01:26.587Z","dependency_job_id":null,"html_url":"https://github.com/coreos/mantle","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreos%2Fmantle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreos%2Fmantle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreos%2Fmantle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coreos%2Fmantle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coreos","download_url":"https://codeload.github.com/coreos/mantle/tar.gz/refs/heads/cl","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250100719,"owners_count":21374991,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coreos","go"],"created_at":"2024-08-02T01:02:50.538Z","updated_at":"2025-04-21T17:32:28.655Z","avatar_url":"https://github.com/coreos.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"\u003cfont size=\"+2\"\u003e\u003cb\u003e:rotating_light: Mantle for Fedora CoreOS and RHEL\nCoreOS has been merged into\n[coreos-assembler](https://github.com/coreos/coreos-assembler/).\n:rotating_light:\n\u003cbr\u003e\nThis `cl` branch is for CoreOS Container Linux.\u003c/b\u003e\u003c/font\u003e\n\n# Mantle: Gluing Container Linux together\n\nThis repository is a collection of utilities for developing Container Linux. Most of the\ntools are for uploading, running, and interacting with Container Linux instances running\nlocally or in a cloud.\n\n## Overview\nMantle is composed of many utilities:\n - `cork` for handling the Container Linux SDK\n - `gangue` for downloading from Google Storage\n - `kola` for launching instances and running tests\n - `kolet` an agent for kola that runs on instances\n - `ore` for interfacing with cloud providers\n - `plume` for releasing Container Linux\n\nAll of the utilities support the `help` command to get a full listing of their subcommands\nand options.\n\n## Tools\n\n### cork\nCork is a tool that helps working with Container Linux images and the SDK.\n\n#### cork create\nDownload and unpack the Container Linux SDK.\n\n`cork create`\n\n#### cork enter\nEnter the SDK chroot, and optionally run a command. The command and its\narguments can be given after `--`.\n\n`cork enter -- repo sync`\n\n#### cork download-image\nDownload a Container Linux image into `$PWD/.cache/images`.\n\n`cork download-image --platform=qemu`\n\n#### Building Container Linux with cork\nSee [Modifying Container Linux](https://coreos.com/os/docs/latest/sdk-modifying-coreos.html) for\nan example of using cork to build a Container Linux image.\n\n### gangue\nGangue is a tool for downloading and verifying files from Google Storage with authenticated requests.\nIt is primarily used by the SDK.\n\n#### gangue get\nGet a file from Google Storage and verify it using GPG.\n\n### kola\nKola is a framework for testing software integration in Container Linux instances\nacross multiple platforms. It is primarily designed to operate within\nthe Container Linux SDK for testing software that has landed in the OS image.\nIdeally, all software needed for a test should be included by building\nit into the image from the SDK.\n\nKola supports running tests on multiple platforms, currently QEMU, GCE,\nAWS, VMware VSphere, Packet, and OpenStack. In the future systemd-nspawn and other\nplatforms may be added.\nLocal platforms do not rely on access to the Internet as a design\nprinciple of kola, minimizing external dependencies. Any network\nservices required get built directly into kola itself. Machines on cloud\nplatforms do not have direct access to the kola so tests may depend on\nInternet services such as discovery.etcd.io or quay.io instead.\n\nKola outputs assorted logs and test data to `_kola_temp` for later\ninspection.\n\nKola is still under heavy development and it is expected that its\ninterface will continue to change.\n\nBy default, kola uses the `qemu` platform with the most recently built image\n(assuming it is run from within the SDK).\n\n#### kola run\nThe run command invokes the main kola test harness. It\nruns any tests whose registered names matches a glob pattern.\n\n`kola run \u003cglob pattern\u003e`\n\n`--blacklist-test` can be used if one or more tests in the pattern should be skipped.\nThis switch may be provided once:\n\n`kola --blacklist-test linux.nfs.v3 run`\n\nmultiple times:\n\n`kola --blacklist-test linux.nfs.v3 --blacklist-test linux.nfs.v4 run`\n\nand can also be used with glob patterns:\n\n`kola --blacklist-test linux.nfs* --blacklist-test crio.* run`\n\n#### kola list\nThe list command lists all of the available tests.\n\n#### kola spawn\nThe spawn command launches Container Linux instances.\n\n#### kola mkimage\nThe mkimage command creates a copy of the input image with its primary console set\nto the serial port (/dev/ttyS0). This causes more output to be logged on the console,\nwhich is also logged in `_kola_temp`. This can only be used with QEMU images and must\nbe used with the `coreos_*_image.bin` image, *not* the `coreos_*_qemu_image.img`.\n\n#### kola bootchart\nThe bootchart command launches an instance then generates an svg of the boot process\nusing `systemd-analyze`.\n\n#### kola updatepayload\nThe updatepayload command launches a Container Linux instance then updates it by\nsending an update to its update_engine. The update is the `coreos_*_update.gz` in the\nlatest build directory.\n\n#### kola subtest parallelization\nSubtests can be parallelized by adding `c.H.Parallel()` at the top of the inline function\ngiven to `c.Run`. It is not recommended to utilize the `FailFast` flag in tests that utilize\nthis functionality as it can have unintended results.\n\n#### kola test namespacing\nThe top-level namespace of tests should fit into one of the following categories:\n1. Groups of tests targeting specific packages/binaries may use that\nnamespace (ex: `docker.*`)\n2. Tests that target multiple supported distributions may use the\n`coreos` namespace.\n3. Tests that target singular distributions may use the distribution's\nnamespace.\n\n#### kola test registration\nRegistering kola tests currently requires that the tests are registered\nunder the kola package and that the test function itself lives within\nthe mantle codebase.\n\nGroups of similar tests are registered in an init() function inside the\nkola package.  `Register(*Test)` is called per test. A kola `Test`\nstruct requires a unique name, and a single function that is the entry\npoint into the test. Additionally, userdata (such as a Container Linux\nConfig) can be supplied. See the `Test` struct in\n[kola/register/register.go](https://github.com/coreos/mantle/tree/master/kola/register/register.go)\nfor a complete list of options.\n\n#### kola test writing\nA kola test is a go function that is passed a `platform.TestCluster` to\nrun code against.  Its signature is `func(platform.TestCluster)`\nand must be registered and built into the kola binary. \n\nA `TestCluster` implements the `platform.Cluster` interface and will\ngive you access to a running cluster of Container Linux machines. A test writer\ncan interact with these machines through this interface.\n\nTo see test examples look under\n[kola/tests](https://github.com/coreos/mantle/tree/master/kola/tests) in the\nmantle codebase.\n\nFor a quickstart see [kola/README.md](/kola/README.md).\n\n#### kola native code\nFor some tests, the `Cluster` interface is limited and it is desirable to\nrun native go code directly on one of the Container Linux machines. This is\ncurrently possible by using the `NativeFuncs` field of a kola `Test`\nstruct. This like a limited RPC interface.\n\n`NativeFuncs` is used similar to the `Run` field of a registered kola\ntest. It registers and names functions in nearby packages.  These\nfunctions, unlike the `Run` entry point, must be manually invoked inside\na kola test using a `TestCluster`'s `RunNative` method. The function\nitself is then run natively on the specified running Container Linux instances.\n\nFor more examples, look at the\n[coretest](https://github.com/coreos/mantle/tree/master/kola/tests/coretest)\nsuite of tests under kola. These tests were ported into kola and make\nheavy use of the native code interface.\n\n#### Manhole\nThe `platform.Manhole()` function creates an interactive SSH session which can\nbe used to inspect a machine during a test.\n\n### kolet\nkolet is run on kola instances to run native functions in tests. Generally kolet\nis not invoked manually.\n\n### ore\nOre provides a low-level interface for each cloud provider. It has commands\nrelated to launching instances on a variety of platforms (gcloud, aws,\nazure, esx, and packet) within the latest SDK image. Ore mimics the underlying\napi for each cloud provider closely, so the interface for each cloud provider\nis different. See each providers `help` command for the available actions.\n\nNote, when uploading to some cloud providers (e.g. gce) the image may need to be packaged\nwith a different --format (e.g. --format=gce) when running `image_to_vm.sh`\n\n### plume\nPlume is the Container Linux release utility. Releases are done in two stages,\neach with their own command: pre-release and release. Both of these commands are idempotent.\n\n#### plume pre-release\nThe pre-release command does as much of the release process as possible without making anything public.\nThis includes uploading images to cloud providers (except those like gce which don't allow us to upload\nimages without making them public).\n\n### plume release\nPublish a new Container Linux release. This makes the images uploaded by pre-release public and uploads\nimages that pre-release could not. It copies the release artifacts to public storage buckets and updates\nthe directory index.\n\n#### plume index\nGenerate and upload index.html objects to turn a Google Cloud Storage\nbucket into a publicly browsable file tree. Useful if you want something\nlike Apache's directory index for your software download repository.\nPlume release handles this as well, so it does not need to be run as part of\nthe release process.\n\n## Platform Credentials\nEach platform reads the credentials it uses from different files. The `aws`, `azure`, `do`, `esx` and `packet`\nplatforms support selecting from multiple configured credentials, call \"profiles\". The examples below\nare for the \"default\" profile, but other profiles can be specified in the credentials files and selected\nvia the `--\u003cplatform-name\u003e-profile` flag:\n```\nkola spawn -p aws --aws-profile other_profile\n```\n\n### aws\n`aws` reads the `~/.aws/credentials` file used by Amazon's aws command-line tool.\nIt can be created using the `aws` command:\n```\n$ aws configure\n```\nTo configure a different profile, use the `--profile` flag\n```\n$ aws configure --profile other_profile\n```\n\nThe `~/.aws/credentials` file can also be populated manually:\n```\n[default]\naws_access_key_id = ACCESS_KEY_ID_HERE\naws_secret_access_key = SECRET_ACCESS_KEY_HERE\n```\n\nTo install the `aws` command in the SDK, run:\n```\nsudo emerge --ask awscli\n```\n\n### azure\n`azure` uses `~/.azure/azureProfile.json`. This can be created using the `az` [command](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli):\n```\n$ az login`\n```\nIt also requires that the environment variable `AZURE_AUTH_LOCATION` points to a JSON file (this can also be set via the `--azure-auth` parameter). The JSON file will require a service provider active directory account to be created.\n\nService provider accounts can be created via the `az` command (the output will contain an `appId` field which is used as the `clientId` variable in the `AZURE_AUTH_LOCATION` JSON):\n```\naz ad sp create-for-rbac\n```\n\nThe client secret can be created inside of the Azure portal when looking at the service provider account under the `Azure Active Directory` service on the `App registrations` tab.\n\nYou can find your subscriptionId \u0026 tenantId in the `~/.azure/azureProfile.json` via:\n```\ncat ~/.azure/azureProfile.json | jq '{subscriptionId: .subscriptions[].id, tenantId: .subscriptions[].tenantId}'\n```\n\nThe JSON file exported to the variable `AZURE_AUTH_LOCATION` should be generated by hand and have the following contents:\n```\n{\n  \"clientId\": \"\u003cservice provider id\u003e\", \n  \"clientSecret\": \"\u003cservice provider secret\u003e\", \n  \"subscriptionId\": \"\u003csubscription id\u003e\", \n  \"tenantId\": \"\u003ctenant id\u003e\", \n  \"activeDirectoryEndpointUrl\": \"https://login.microsoftonline.com\", \n  \"resourceManagerEndpointUrl\": \"https://management.azure.com/\", \n  \"activeDirectoryGraphResourceId\": \"https://graph.windows.net/\", \n  \"sqlManagementEndpointUrl\": \"https://management.core.windows.net:8443/\", \n  \"galleryEndpointUrl\": \"https://gallery.azure.com/\", \n  \"managementEndpointUrl\": \"https://management.core.windows.net/\"\n}\n\n```\n\n### do\n`do` uses `~/.config/digitalocean.json`. This can be configured manually:\n```\n{\n    \"default\": {\n        \"token\": \"token goes here\"\n    }\n}\n```\n\n### esx\n`esx` uses `~/.config/esx.json`. This can be configured manually:\n```\n{\n    \"default\": {\n        \"server\": \"server.address.goes.here\",\n        \"user\": \"user.goes.here\",\n        \"password\": \"password.goes.here\"\n    }\n}\n```\n\n### gce\n`gce` uses the `~/.boto` file. When the `gce` platform is first used, it will print\na link that can be used to log into your account with gce and get a verification code\nyou can paste in. This will populate the `.boto` file.\n\nSee [Google Cloud Platform's Documentation](https://cloud.google.com/storage/docs/boto-gsutil)\nfor more information about the `.boto` file.\n\n### openstack\n`openstack` uses `~/.config/openstack.json`. This can be configured manually:\n```\n{\n    \"default\": {\n        \"auth_url\": \"auth url here\",\n        \"tenant_id\": \"tenant id here\",\n        \"tenant_name\": \"tenant name here\",\n        \"username\": \"username here\",\n        \"password\": \"password here\",\n        \"user_domain\": \"domain id here\",\n        \"floating_ip_pool\": \"floating ip pool here\",\n        \"region_name\": \"region here\"\n    }\n}\n```\n\n`user_domain` is required on some newer versions of OpenStack using Keystone V3 but is optional on older versions. `floating_ip_pool` and `region_name` can be optionally specified here to be used as a default if not specified on the command line.\n\n### packet\n`packet` uses `~/.config/packet.json`. This can be configured manually:\n```\n{\n\t\"default\": {\n\t\t\"api_key\": \"your api key here\",\n\t\t\"project\": \"project id here\"\n\t}\n}\n```\n\n### qemu\n`qemu` is run locally and needs no credentials, but does need to be run as root.\n\n### qemu-unpriv\n`qemu-unpriv` is run locally and needs no credentials. It has a restricted set of functionality compared to the `qemu` platform, such as:\n\n- Single node only, no machine to machine networking\n- DHCP provides no data (forces several tests to be disabled)\n- No [Local cluster](platform/local/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoreos%2Fmantle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoreos%2Fmantle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoreos%2Fmantle/lists"}