{"id":17435610,"url":"https://github.com/jnsgruk/concierge","last_synced_at":"2025-04-16T03:31:17.671Z","repository":{"id":257821476,"uuid":"870661681","full_name":"jnsgruk/concierge","owner":"jnsgruk","description":"An opinionated utility for provisioning charm development and testing machines.","archived":false,"fork":false,"pushed_at":"2025-03-06T08:45:12.000Z","size":735,"stargazers_count":7,"open_issues_count":3,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T04:42:14.296Z","etag":null,"topics":["automation","development","go","juju","provisioning","spread","testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jnsgruk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-10T12:50:39.000Z","updated_at":"2025-03-06T08:45:16.000Z","dependencies_parsed_at":"2024-10-11T07:55:09.957Z","dependency_job_id":"596eed28-af1a-46b6-bcf7-713dcc9253df","html_url":"https://github.com/jnsgruk/concierge","commit_stats":{"total_commits":89,"total_committers":2,"mean_commits":44.5,"dds":0.4606741573033708,"last_synced_commit":"5e751ecf3e9d47a9303402718af433aa1802b9e1"},"previous_names":["jnsgruk/concierge"],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnsgruk%2Fconcierge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnsgruk%2Fconcierge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnsgruk%2Fconcierge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jnsgruk%2Fconcierge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jnsgruk","download_url":"https://codeload.github.com/jnsgruk/concierge/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249191120,"owners_count":21227505,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","development","go","juju","provisioning","spread","testing"],"created_at":"2024-10-17T10:00:47.224Z","updated_at":"2025-04-16T03:31:17.663Z","avatar_url":"https://github.com/jnsgruk.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg width=\"250px\" src=\".github/concierge.png\" alt=\"concierge logo\"\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003econcierge\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://snapcraft.io/concierge\"\u003e\u003cimg src=\"https://snapcraft.io/concierge/badge.svg\" alt=\"Snap Status\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/jnsgruk/concierge/actions/workflows/release.yaml\"\u003e\u003cimg src=\"https://github.com/jnsgruk/concierge/actions/workflows/release.yaml/badge.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n`concierge` is an opinionated utility for provisioning charm development and testing machines.\n\nIts role is to ensure that a given machine has the relevant \"craft\" tools and providers installed,\nthen bootstrap a Juju controller onto each of the providers. Additionally, it can install selected\ntools from the [snap store](https://snapcraft.io) or the Ubuntu archive.\n\n`concierge` also provides the facility to \"restore\" a machine, performing the opposite to \"prepare\"\nmeaning that, for example, any snaps that would have been installed by `concierge`, would then be\nremoved.\n\n\u003e [!IMPORTANT]\n\u003e Take care with `concierge restore`. If the machine already had a given snap or configuration\n\u003e prior to running `concierge prepare`, this will not be taken into account during `restore`.\n\u003e Running `concierge restore` is the literal opposite of `concierge prepare`, so any packages,\n\u003e files or configuration that would normally be created during `prepare` will be removed.\n\n## Installation\n\nThe easiest way to consume `concierge` is using the [Snap](https://snapcraft.io/concierge):\n\n```shell\nsudo snap install --classic concierge\n```\n\nOr, you can install `concierge` with the `go install` command:\n\n```shell\ngo install github.com/jnsgruk/concierge@latest\n```\n\nOr you can clone, build and run like so:\n\n```shell\ngit clone https://github.com/jnsgruk/concierge\ncd concierge\ngo build\nsudo ./concierge -h\n```\n\n## Usage\n\nThe output of `concierge --help` can be seen below.\n\n```\nconcierge is an opinionated utility for provisioning charm development and testing machines.\n\nIts role is to ensure that a given machine has the relevant \"craft\" tools and providers installed,\nthen bootstrap a Juju controller onto each of the providers. Additionally, it can install selected\ntools from the [snap store](https://snapcraft.io) or the Ubuntu archive.\n\nUsage:\n  concierge [flags]\n  concierge [command]\n\nAvailable Commands:\n  completion  Generate the autocompletion script for the specified shell\n  help        Help about any command\n  prepare     Provision the machine according to the configuration.\n  restore     Run the reverse of `concierge prepare`.\n  status      Report the status of `concierge` on the machine.\n\nFlags:\n  -h, --help      help for concierge\n      --trace     enable trace logging\n  -v, --verbose   enable verbose logging\n      --version   version for concierge\n\nUse \"concierge [command] --help\" for more information about a command.\n```\n\nSome flags can be set by environment variable, and if specified by flag and environment variable,\nthe environment variable version will always take precedent. The equivalents are:\n\n|            Flag            |              Env Var               |\n| :------------------------: | :--------------------------------: |\n|      `--disable-juju`      |      `CONCIERGE_DISABLE_JUJU`      |\n|      `--juju-channel`      |      `CONCIERGE_JUJU_CHANNEL`      |\n|      `--k8s-channel`       |      `CONCIERGE_K8S_CHANNEL`       |\n|    `--microk8s-channel`    |    `CONCIERGE_MICROK8S_CHANNEL`    |\n|      `--lxd-channel`       |      `CONCIERGE_LXD_CHANNEL`       |\n|   `--charmcraft-channel`   |   `CONCIERGE_CHARMCRAFT_CHANNEL`   |\n|   `--snapcraft-channel`    |   `CONCIERGE_SNAPCRAFT_CHANNEL`    |\n|   `--rockcraft-channel`    |   `CONCIERGE_ROCKCRAFT_CHANNEL`    |\n| `--google-credential-file` | `CONCIERGE_GOOGLE_CREDENTIAL_FILE` |\n|      `--extra-snaps`       |      `CONCIERGE_EXTRA_SNAPS`       |\n|       `--extra-debs`       |       `CONCIERGE_EXTRA_DEBS`       |\n\n### Command Examples\n\nThe best source of examples for how to invoke `concierge` can be found in the\n[tests](./tests/) directory, but otherwise:\n\n1. Run `concierge` using the `dev` preset, adding one additional snap, using CLI flags:\n\n```bash\nsudo concierge prepare -p dev --extra-snaps node/22/stable\n```\n\n2. Run `concierge` using the `dev` preset, overriding the Juju channel:\n\n```bash\nexport CONCIERGE_JUJU_CHANNEL=3.6/beta\nsudo concierge prepare -p dev\n```\n\n## Configuration\n\n### Presets\n\n`concierge` comes with a number of presets that are likely to serve most charm development needs:\n\n| Preset Name | Included                                                                  |\n| :---------: | :------------------------------------------------------------------------ |\n|  `crafts`   | `lxd`, `snapcraft`, `charmcraft`, `rockcraft`                             |\n|    `dev`    | `juju`, `k8s`, `lxd`, `snapcraft`, `charmcraft`, `rockcraft`, `jhack`     |\n|    `k8s`    | `juju`, `k8s`, `lxd`, `rockcraft`, `charmcraft`                           |\n| `microk8s`  | `juju`, `microk8s`, `lxd`, `rockcraft`, `charmcraft`                      |\n|  `machine`  | `juju`, `lxd`, `snapcraft`, `charmcraft`                                  |\n\nNote that in the `microk8s`/`k8s` presets, while `lxd` is installed, it is not bootstrapped. It is\ninstalled and initialised with enough config such that `charmcraft` can use it as a build backend.\n\n### Config File\n\nIf the presets do not meet your needs, you can create your own config file to instruct `concierge`\nhow to provision your machine.\n\n`concierge` takes configuration in the form of a YAML file named `concierge.yaml` in the current\nworking directory.\n\n#### Schema\n\n```yaml\n# (Optional): Target Juju configuration.\njuju:\n  # (Optional): Disable installation of Juju (and therefore all bootstrapping).\n  disable: true | false\n  # (Optional): Channel from which to install Juju.\n  channel: \u003cchannel\u003e\n  # (Optional): A map of model-defaults to set when bootstrapping *all* Juju controllers.\n  model-defaults:\n    \u003cmodel-default\u003e: \u003cvalue\u003e\n  # (Optional): A map of bootstrap-constraints to set when bootstrapping *all* Juju controllers.\n  bootstrap-constraints:\n    \u003cbootstrap-constraint\u003e: \u003cvalue\u003e\n\n# (Required): Define the providers to be installed and bootstrapped.\nproviders:\n  # (Optional) MicroK8s provider configuration.\n  microk8s:\n    # (Optional) Enable or disable MicroK8s.\n    enable: true | false\n    # (Optional) Whether or not to bootstrap a controller onto MicroK8s.\n    bootstrap: true | false\n    # (Optional): A map of model-defaults to set when bootstrapping the Juju controller.\n    model-defaults:\n      \u003cmodel-default\u003e: \u003cvalue\u003e\n    # (Optional): A map of bootstrap-constraints to set when bootstrapping the Juju controller.\n    bootstrap-constraints:\n      \u003cbootstrap-constraint\u003e: \u003cvalue\u003e\n    # (Optional): Channel from which to install MicroK8s.\n    channel: \u003cchannel\u003e\n    # (Optional): MicroK8s addons to enable.\n    addons:\n      - \u003caddon\u003e[:\u003cparams\u003e]\n\n  # (Optional) K8s provider configuration.\n  k8s:\n    # (Optional) Enable or disable K8s.\n    enable: true | false\n    # (Optional) Whether or not to bootstrap a controller onto K8s.\n    bootstrap: true | false\n    # (Optional): Channel from which to install K8s.\n    channel: \u003cchannel\u003e\n    # (Optional): A map of model-defaults to set when bootstrapping the Juju controller.\n    model-defaults:\n      \u003cmodel-default\u003e: \u003cvalue\u003e\n    # (Optional): A map of bootstrap-constraints to set when bootstrapping the Juju controller.\n    bootstrap-constraints:\n      \u003cbootstrap-constraint\u003e: \u003cvalue\u003e\n    # (Optional): K8s features to configure.\n    features:\n      \u003cfeature\u003e:\n        \u003ckey\u003e: \u003cvalue\u003e\n\n  # (Optional) LXD provider configuration.\n  lxd:\n    # (Optional) Enable or disable LXD.\n    enable: true | false\n    # (Optional) Whether or not to bootstrap a controller onto LXD.\n    bootstrap: true | false\n    # (Optional): Channel from which to install LXD.\n    channel: \u003cchannel\u003e\n    # (Optional): A map of model-defaults to set when bootstrapping the Juju controller.\n    model-defaults:\n      \u003cmodel-default\u003e: \u003cvalue\u003e\n    # (Optional): A map of bootstrap-constraints to set when bootstrapping the Juju controller.\n    bootstrap-constraints:\n      \u003cbootstrap-constraint\u003e: \u003cvalue\u003e\n\n  # (Optional) Google provider configuration.\n  google:\n    # (Optional) Enable or disable the Google provider.\n    enable: true | false\n    # (Optional) Whether or not to bootstrap a controller onto Google Cloud.\n    bootstrap: true | false\n    # (Optional): File containing credentials for Google cloud.\n    # See below note on the credentials file format.\n    credentials-file: \u003cpath\u003e\n    # (Optional): A map of model-defaults to set when bootstrapping the Juju controller.\n    model-defaults:\n      \u003cmodel-default\u003e: \u003cvalue\u003e\n    # (Optional): A map of bootstrap-constraints to set when bootstrapping the Juju controller.\n    bootstrap-constraints:\n      \u003cbootstrap-constraint\u003e: \u003cvalue\u003e\n\n# (Optional) Additional host configuration.\nhost:\n  # (Optional) List of apt packages to install on the host.\n  packages:\n    - \u003cpackage name\u003e\n  # (Optional) Map of snap packages to install on the host.\n  snaps:\n    \u003csnap name\u003e:\n      # (Required) Channel from which to install the snap.\n      channel: \u003cchannel\u003e\n      # (Optional) List of snap connections to form.\n      connections:\n        - \u003csnap\u003e:\u003cplug-interface\u003e\n        - \u003csnap\u003e:\u003cplug-interface\u003e \u003csnap\u003e:\u003cplug-interface\u003e\n```\n\n#### Providing Credentials Files\n\nJuju has some \"built-in\" clouds for which it can obtain credentials automatically, such as LXD and MicroK8s. Other clouds require credentials for the bootstrap process.\n\nConcierge handles this with the `credentials-file` option for supported providers.\n\nJuju's credentials are specified in `~/.local/share/juju/credentials.yaml`, in the following format:\n\n```yaml\ncredentials:\n  \u003ccloud-name\u003e:\n    \u003ccredential-name\u003e:\n      key: value\n      key: value\n```\n\nFor example, a pre-configured `credentials.yaml` might look like so where Google Cloud had already been configured:\n\n```yaml\ncredentials:\n  google:\n    mycred:\n      auth-type: oauth2\n      client-email: juju-gce-1-sa@myname.iam.gserviceaccount.com\n      client-id: \"1234567891234\"\n      private-key: |\n        -----BEGIN PRIVATE KEY-----\n        deadbeef\n        -----END PRIVATE KEY-----\n      project-id: foobar\n```\n\nWhen providing the path to a `credentials-file` for `concierge`, it should contain _only_ the details for a specific credential, so the example above would require a file like so:\n\n```yaml\nauth-type: oauth2\nclient-email: juju-gce-1-sa@myname.iam.gserviceaccount.com\nclient-id: \"1234567891234\"\nprivate-key: |\n  -----BEGIN PRIVATE KEY-----\n  deadbeef\n  -----END PRIVATE KEY-----\nproject-id: foobar\n```\n\nIf you already have a `credentials.yaml` pre-populated with credentials, you can use `yq` to build the file for you, for example:\n\n```bash\ncat ~/.local/share/juju/credentials.yaml | yq -r '.credentials.google.mycred' \u003e google-creds.yaml\n```\n\nIn the above example `google-creds.yaml` would be valid for the `credentials-file` option.\n\n#### Example Config\n\nAn example config file can be seen below:\n\n```yaml\njuju:\n  channel: 3.5/stable\n  model-defaults:\n    test-mode: \"true\"\n    automatically-retry-hooks: \"false\"\n  bootstrap-constraints:\n    arch: amd64\n\nproviders:\n  microk8s:\n    enable: true\n    bootstrap: true\n    channel: 1.31-strict/stable\n    addons:\n      - hostpath-storage\n      - dns\n      - rbac\n      - metallb:10.64.140.43-10.64.140.49\n\n  k8s:\n    enable: true\n    bootstrap: true\n    channel: 1.32-classic/stable\n    features:\n      local-storage:\n      load-balancer:\n        l2-mode: true\n        cidrs: 10.64.140.43/32\n    bootstrap-constraints:\n      root-disk: 2G\n\n  lxd:\n    enable: true\n    bootstrap: false\n    channel: latest/stable\n\n  google:\n    enable: true\n    bootstrap: false\n    credentials-file: /home/ubuntu/google-credentials.yaml\n\nhost:\n  packages:\n    - python3-pip\n    - python3-venv\n  snaps:\n    astral-uv:\n      channel: latest/stable\n    charmcraft:\n      channel: latest/stable\n    jhack:\n      channel: latest/edge\n      connections:\n        - jhack:dot-local-share-juju\n```\n\n## Development / HACKING\n\nThis project uses [goreleaser](https://goreleaser.com/) to build and release, and `spread` for\nintegration testing,\n\nYou can get started by just using Go, or with `goreleaser`:\n\n```shell\n# Clone the repository\ngit clone https://github.com/jnsgruk/concierge\ncd concierge\n\n# Build/run with Go\ngo run .\n\n# Run the unit tests\ngo test ./...\n\n# Build a snapshot release with goreleaser (output in ./dist)\ngoreleaser build --clean --snapshot\n```\n\n### Testing\n\nMost of the code within tries to call a shell command, or manipulate the system in some way, which\nmakes unit testing much more awkward. As a result, the majority of the testing is done with\n[`spread`](https://github.com/canonical/spread).\n\nCurrently, there are two supported backends - tests can either be run in LXD virtual machines, or\non a pre-provisioned server (such as a Github Actions runner or development VM).\n\nTo show the available integration tests, you can:\n\n```bash\n$ spread -list lxd:\nlxd:ubuntu-24.04:tests/extra-debs\nlxd:ubuntu-24.04:tests/extra-packages-config-file\nlxd:ubuntu-24.04:tests/extra-snaps\n# ...\n```\n\nFrom there, you can either run all of the tests, or a selection:\n\n```bash\n# Run all of the tests\n$ spread -v lxd:\n# Run a particular test\n$ spread -v lxd:ubuntu-24.04:tests/juju-model-defaults\n```\n\nTo run any of the tests on a locally provisioned machine, use the `github-ci` backend, e.g.\n\n```bash\n# List available tests\n$ spread --list github-ci:\n# Run all of the tests\n$ spread -v github-ci:\n# Run a particular test\n$ spread -v github-ci:ubuntu-24.04:tests/juju-model-defaults\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjnsgruk%2Fconcierge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjnsgruk%2Fconcierge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjnsgruk%2Fconcierge/lists"}