{"id":20111443,"url":"https://github.com/openstack/charm-ceph-osd","last_synced_at":"2025-05-06T10:33:37.213Z","repository":{"id":53720382,"uuid":"52858687","full_name":"openstack/charm-ceph-osd","owner":"openstack","description":"Juju Charm - Ceph OSD. Mirror of code maintained at opendev.org.","archived":false,"fork":false,"pushed_at":"2024-08-13T14:02:09.000Z","size":2521,"stargazers_count":13,"open_issues_count":0,"forks_count":22,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-08-14T09:55:24.925Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://opendev.org/openstack/charm-ceph-osd","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openstack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-03-01T07:53:31.000Z","updated_at":"2024-08-14T09:55:24.925Z","dependencies_parsed_at":"2023-02-17T23:15:33.844Z","dependency_job_id":"d6faa42f-233c-4377-a19b-b94af988e91e","html_url":"https://github.com/openstack/charm-ceph-osd","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openstack%2Fcharm-ceph-osd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openstack%2Fcharm-ceph-osd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openstack%2Fcharm-ceph-osd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openstack%2Fcharm-ceph-osd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openstack","download_url":"https://codeload.github.com/openstack/charm-ceph-osd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224499851,"owners_count":17321605,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T18:16:04.867Z","updated_at":"2024-11-13T18:16:05.655Z","avatar_url":"https://github.com/openstack.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Overview\n\n[Ceph][ceph-upstream] is a unified, distributed storage system designed for\nexcellent performance, reliability, and scalability.\n\nThe ceph-osd charm deploys the Ceph object storage daemon (OSD) and manages its\nvolumes. It is used in conjunction with the [ceph-mon][ceph-mon-charm] charm.\nTogether, these charms can scale out the amount of storage available in a Ceph\ncluster.\n\n# Usage\n\n## Configuration\n\nThis section covers common and/or important configuration options. See file\n`config.yaml` for the full list of options, along with their descriptions and\ndefault values. A YAML file (e.g. `ceph-osd.yaml`) is often used to store\nconfiguration options. See the [Juju documentation][juju-docs-config-apps] for\ndetails on configuring applications.\n\n#### `bluestore`\n\nThe `bluestore` option specifies whether the\n[BlueStore][upstream-ceph-bluestore] storage backend is used for all OSD\ndevices. The feature is enabled by default (value 'True'). If set to 'True',\nthis option overrides the `osd-format` option as BlueStore does not use a\ntraditional filesystem.\n\n\u003e **Important**: This option has no effect unless Ceph Luminous (or greater) is\n  in use.\n\n#### `customize-failure-domain`\n\nThe `customize-failure-domain` option determines how a Ceph CRUSH map is\nconfigured.\n\nA value of 'false' (the default) will lead to a map that will replicate data\nacross hosts (implemented as [Ceph bucket type][upstream-ceph-buckets] 'host').\nWith a value of 'true' all MAAS-defined zones will be used to generate a map\nthat will replicate data across Ceph availability zones (implemented as bucket\ntype 'rack').\n\nThis option is also supported by the ceph-mon charm. Its value must be the same\nfor both charms.\n\n#### `osd-devices`\n\nThe `osd-devices` option lists what block devices can be used for OSDs across\nthe cluster. Devices that are listed in this option, but do not exist, will\nbe ignored.\n\nSee section 'Storage devices' for an elaboration on this fundamental topic.\n\n#### `osd-format`\n\nThe `osd-format` option specifies what filesystem to use for all OSD devices\n('xfs' or 'ext4'). The default value is 'xfs'. This option only applies when\nCeph Luminous (or greater) is in use and option `bluestore` is set to 'False'.\n\n#### `source`\n\nThe `source` option states the software sources. A common value is an OpenStack\nUCA release (e.g. 'cloud:xenial-queens' or 'cloud:bionic-ussuri'). See [Ceph\nand the UCA][cloud-archive-ceph]. The underlying host's existing apt sources\nwill be used if this option is not specified (this behaviour can be explicitly\nchosen by using the value of 'distro').\n\n### Storage devices\n\nA storage device is destined as an OSD (Object Storage Device). There can be\nmultiple OSDs per storage node (ceph-osd unit).\n\nThe list of all possible storage devices for the cluster is defined by the\n`osd-devices` option. The below examples can be used in the `ceph-osd.yaml`\nconfiguration file.\n\nBlock devices (regular),\n\n    ceph-osd:\n      osd-devices: /dev/vdb /dev/vdc /dev/vdd\n\nEach regular block device must be an absolute path to a device node.\n\nBlock devices (Juju storage),\n\n    ceph-osd:\n      storage:\n        osd-devices: cinder,20G\n\nSee the [Juju documentation][juju-docs-storage] for guidance on implementing\nJuju storage.\n\nDirectory-backed OSDs,\n\n    ceph-osd:\n      storage:\n        osd-devices: /var/tmp/osd-1\n\n\u003e **Note**: OSD directories can no longer be created starting with Ceph\n  Nautilus. Existing OSD directories will continue to function after an upgrade\n  to Nautilus.\n\nThe list defined by option `osd-devices` may affect newly added ceph-osd units\nas well as existing units (the option may be modified after units have been\nadded). The charm will attempt to activate as Ceph storage any listed device\nthat is visible by the unit's underlying machine. To prevent the activation of\nvolumes on existing units the `blacklist-add-disk` action may be used.\n\nThe configuration option is modified in the usual way. For instance, to have it\nconsist solely of devices '/dev/sdb' and '/dev/sdc':\n\n    juju config ceph-osd osd-devices='/dev/sdb /dev/sdc'\n\nThe charm will go into a blocked state (visible in `juju status` output) if it\ndetects pre-existing data on a device. In this case the operator can either\ninstruct the charm to ignore the disk (action `blacklist-add-disk`) or to have\nit purge all data on the disk (action `zap-disk`).\n\n\u003e **Important**: The recommended minimum number of OSDs in the cluster is three\n  and this is what the ceph-mon charm expects (the cluster will not form with a\n  lesser number). See option `expected-osd-count` in the ceph-mon charm to\n  overcome this but beware that going below three is not a supported\n  configuration.\n\n## Deployment\n\nA cloud with three MON nodes is a typical design whereas three OSDs are\nconsidered the minimum. For example, to deploy a Ceph cluster consisting of\nthree OSDs (one per ceph-osd unit) and three MONs:\n\n    juju deploy -n 3 --config ceph-osd.yaml ceph-osd\n    juju deploy -n 3 --to lxd:0,lxd:1,lxd:2 ceph-mon\n    juju integrate ceph-osd:mon ceph-mon:osd\n\nHere, a containerised MON is running alongside each storage node. We've assumed\nthat the machines spawned in the first command are assigned IDs of 0, 1, and 2.\n\n\u003e **Note**: Refer to the [Install OpenStack][cdg-install-openstack] page in the\n  OpenStack Charms Deployment Guide for instructions on installing the ceph-osd\n  application for use with OpenStack.\n\nFor each ceph-osd unit, the ceph-osd charm will scan for all the devices\nconfigured via the `osd-devices` option and attempt to assign to it all of the\nones it finds. The cluster's initial pool of available storage is the \"sum\" of\nall these assigned devices.\n\n## Network spaces\n\nThis charm supports the use of Juju [network spaces][juju-docs-spaces] (Juju\n`v.2.0`). This feature optionally allows specific types of the application's\nnetwork traffic to be bound to subnets that the underlying hardware is\nconnected to.\n\n\u003e **Note**: Spaces must be configured in the backing cloud prior to deployment.\n\nThe ceph-osd charm exposes the following Ceph traffic types (bindings):\n\n* 'public' (front-side)\n* 'cluster' (back-side)\n\nFor example, providing that spaces 'data-space' and 'cluster-space' exist, the\ndeploy command above could look like this:\n\n    juju deploy --config ceph-osd.yaml -n 3 ceph-osd \\\n       --bind \"public=data-space cluster=cluster-space\"\n\nAlternatively, configuration can be provided as part of a bundle:\n\n```yaml\n    ceph-osd:\n      charm: cs:ceph-osd\n      num_units: 1\n      bindings:\n        public: data-space\n        cluster: cluster-space\n```\n\nRefer to the [Ceph Network Reference][ceph-docs-network-ref] to learn about the\nimplications of segregating Ceph network traffic.\n\n\u003e **Note**: Existing ceph-osd units configured with the `ceph-public-network`\n  or `ceph-cluster-network` options will continue to honour them. Furthermore,\n  these options override any space bindings, if set.\n\n## AppArmor profiles\n\nAlthough AppArmor is not enabled for Ceph by default, an AppArmor profile can\nbe generated by the charm by assigning a value of 'complain', 'enforce', or\n'disable' (the default) to option `aa-profile-mode`.\n\n\u003e **Caution**: Enabling an AppArmor profile is disruptive to a running Ceph\n  cluster as all ceph-osd processes must be restarted.\n\nThe new profile has a narrow supported use case, and it should always be\nverified in pre-production against the specific configurations and topologies\nintended for production.\n\nThe profiles generated by the charm should **not** be used in the following\nscenarios:\n\n* On any version of Ubuntu older than 16.04\n* On any version of Ceph older than Luminous\n* When OSD journal devices are in use\n* When Ceph BlueStore is enabled\n\n## Block device encryption\n\nThe ceph-osd charm supports encryption for OSD volumes that are backed by block\ndevices. To use Ceph's native key management framework, available since Ceph\nJewel, set option `osd-encrypt` for the ceph-osd charm:\n\n```yaml\n    ceph-osd:\n      osd-encrypt: True\n```\n\nHere, dm-crypt keys are stored in the MON sub-cluster.\n\nAlternatively, since Ceph Luminous, encryption keys can be stored in Vault,\nwhich is deployed and initialised via the [vault][vault-charm] charm. Set\noptions `osd-encrypt` and `osd-encrypt-keymanager` for the ceph-osd charm:\n\n```yaml\n    ceph-osd:\n      osd-encrypt: True\n      osd-encrypt-keymanager: vault\n```\n\n\u003e **Important**: Post deployment configuration will only affect block devices\n  associated with **new** ceph-osd units.\n\n## Actions\n\nThis section covers Juju [actions][juju-docs-actions] supported by the charm.\nActions allow specific operations to be performed on a per-unit basis. To\ndisplay action descriptions run `juju actions --schema ceph-osd`. If the charm\nis not deployed then see file `actions.yaml`.\n\n* `add-disk`\n* `blacklist-add-disk`\n* `blacklist-remove-disk`\n* `get-availibility-zone`\n* `list-disks`\n* `osd-in`\n* `osd-out`\n* `security-checklist`\n* `start`\n* `stop`\n* `zap-disk`\n\n## Working with OSDs\n\n### Set OSDs to 'out'\n\nUse the `osd-out` action to set OSD volumes on a unit to 'out'.\n\n\u003e **Warning**: This action has the potential of impacting your cluster\n  significantly. The [Ceph documentation][ceph-docs-removing-osds] on this\n  topic is considered essential reading.\n\nUnless the cluster itself is set to 'noout' this action will cause Ceph to\nrebalance data by migrating PGs out of the affected OSDs and onto OSDs\navailable on other units. The impact is twofold:\n\n1. The available space on the remaining OSDs is reduced. Not only is there less\n   space for future workloads but there is a danger of exceeding the cluster's\n   storage capacity.\n1. The traffic and CPU load on the cluster is increased.\n\n\u003e **Note**: It has been reported that setting OSDs to 'out' may cause some PGs\n  to get stuck in the 'active+remapped' state. This is an upstream issue.\n\nThe [ceph-mon][ceph-mon-charm] charm has an action called `set-noout` that sets\n'noout' for the cluster.\n\nIt may be perfectly fine to have data rebalanced. The decisive factor is\nwhether the OSDs are being paused temporarily (e.g. the underlying machine is\nscheduled for maintenance) or whether they are being removed from the cluster\ncompletely (e.g. the storage hardware is reaching EOL).\n\nExamples:\n\n    # Set OSDs '0' and '1' to 'out' on unit `ceph-osd/4`\n    juju run ceph-osd/4 osd-out osds=osd.0,osd.1\n\n    # Set all OSDs to 'out' on unit `ceph-osd/2`\n    juju run ceph-osd/2 osd-out osds=all\n\n### Set OSDs to 'in'\n\nUse the `osd-in` action to set OSD volumes on a unit to 'in'.\n\nThe `osd-in` action is reciprocal to the `osd-out` action. The OSDs are set to\n'in'. It is typically used when the `osd-out` action was used in conjunction\nwith the cluster 'noout' flag.\n\nExamples:\n\n    # Set OSDs '0' and '1' to 'in' on unit `ceph-osd/4`\n    juju run ceph-osd/4 osd-in osds=osd.0,osd.1\n\n    # Set all OSDs to 'in' on unit `ceph-osd/2`\n    juju run ceph-osd/2 osd-in osds=all\n\n### Stop and start OSDs\n\nUse the `stop` and `start` actions to stop and start OSD daemons on a unit.\n\n\u003e **Important**: These actions are not available on the 'trusty' series due to\n  the reliance on `systemd`.\n\nExamples:\n\n    # Stop services 'ceph-osd@0' and 'ceph-osd@1' on unit `ceph-osd/4`\n    juju run ceph-osd/4 stop osds=0,1\n\n    # Start all ceph-osd services on unit `ceph-osd/2`\n    juju run ceph-osd/2 start osds=all\n\n\u003e **Note**: Stopping an OSD daemon will put the associated unit into a blocked\n  state.\n\n## Working with disks\n\n### List disks\n\nUse the `list-disks` action to list disks known to a unit.\n\nThe action lists the unit's block devices by categorising them in three ways:\n\n* `disks`: visible (known by udev), unused (not mounted), and not designated as\n  an OSD journal (via the `osd-journal` configuration option)\n\n* `blacklist`: like `disks` but blacklisted (see action `blacklist-add-disk`)\n\n* `non-pristine`: like `disks` but not eligible for use due to the presence of\n  existing data\n\nExample:\n\n    # List disks on unit `ceph-osd/4`\n    juju run ceph-osd/4 list-disks\n\n### Add a disk\n\nUse the `add-disk` action to add a disk to a unit.\n\nA ceph-osd unit is automatically assigned OSD volumes based on the current\nvalue of the `osd-devices` application option. The `add-disk` action allows the\noperator to manually add OSD volumes (for disks that are not listed by\n`osd-devices`) to an existing unit.\n\n**Parameters**\n\n\u003c!-- The next line has two trailing spaces. --\u003e\n\n* `osd-devices` (required)  \n  A space-separated list of devices to format and initialise as OSD volumes.\n\n\u003c!-- The next line has two trailing spaces. --\u003e\n\n* `bucket`  \n  The name of a Ceph bucket to add these devices to.\n\nExample:\n\n    # Add disk /dev/vde on unit `ceph-osd/4`\n    juju run ceph-osd/4 add-disk osd-devices=/dev/vde\n\n### Blacklist a disk\n\nUse the `blacklist-add-disk` action to add a disk to a unit's blacklist.\n\nThe action allows the operator to add disks (that are visible to the unit's\nunderlying machine) to the unit's blacklist. A blacklisted device will not be\ninitialised as an OSD volume when the value of the `osd-devices` application\noption changes. This action does not prevent a device from being activated via\nthe `add-disk` action.\n\nUse the `list-disks` action to list the unit's blacklist entries.\n\n\u003e **Important**: This action and blacklist do not have any effect on current\n  OSD volumes.\n\n**Parameters**\n\n\u003c!-- The next line has two trailing spaces. --\u003e\n\n* `osd-devices` (required)  \n  A space-separated list of devices to add to a unit's blacklist.\n\nExample:\n\n    # Blacklist disks /dev/vda and /dev/vdf on unit `ceph-osd/0`\n    juju run ceph-osd/0 \\\n       blacklist-add-disk osd-devices='/dev/vda /dev/vdf'\n\n### Un-blacklist a disk\n\nUse the `blacklist-remove-disk` action to remove a disk from a unit's\nblacklist.\n\n**Parameters**\n\n\u003c!-- The next line has two trailing spaces. --\u003e\n\n* `osd-devices` (required)  \n  A space-separated list of devices to remove from a unit's blacklist.\n\nEach device should have an existing entry in the unit's blacklist. Use the\n`list-disks` action to list the unit's blacklist entries.\n\nExample:\n\n    # Un-blacklist disk /dev/vdb on unit `ceph-osd/1`\n    juju run ceph-osd/1 \\\n       blacklist-remove-disk osd-devices=/dev/vdb\n\n### Zap a disk\n\nUse the `zap-disk` action to purge a disk of all data.\n\nIn order to prevent unintentional data loss, the charm will not use a disk that\ncontains data. To forcibly make a disk available, the `zap-disk` action can be\nused. Due to the destructive nature of this action the `i-really-mean-it`\noption must be passed. This action is normally followed by the `add-disk`\naction.\n\n**Parameters**\n\n\u003c!-- The next line has two trailing spaces. --\u003e\n\n* `devices` (required)  \n  A space-separated list of devices to be recycled.\n\n\u003c!-- The next line has two trailing spaces. --\u003e\n\n* `i-really-mean-it` (required)  \n  A boolean option for confirming the action.\n\nExample:\n\n    # Zap disk /dev/vdc on unit `ceph-osd/3`\n    juju run ceph-osd/3 \\\n       zap-disk i-really-mean-it=true devices=/dev/vdc\n\n\u003e **Note**: The `zap-disk` action cannot be run on a mounted device, an active\n  BlueStore device, or an encrypted device. There are also issues with\n  LVM-backed volumes (see [LP #1858519][lp-bug-1858519]).\n\n# Documentation\n\nThe OpenStack Charms project maintains two documentation guides:\n\n* [OpenStack Charm Guide][cg]: for project information, including development\n  and support notes\n* [OpenStack Charms Deployment Guide][cdg]: for charm usage information\n\nSee also the [Charmed Ceph documentation][charmed-ceph-docs].\n\n# Bugs\n\nPlease report bugs on [Launchpad][lp-bugs-charm-ceph-osd].\n\n\u003c!-- LINKS --\u003e\n\n[ceph-upstream]: https://ceph.io\n[cg]: https://docs.openstack.org/charm-guide\n[cdg]: https://docs.openstack.org/project-deploy-guide/charm-deployment-guide\n[ceph-mon-charm]: https://jaas.ai/ceph-mon\n[vault-charm]: https://jaas.ai/vault\n[charmed-ceph-docs]: https://ubuntu.com/ceph/docs\n[juju-docs-storage]: https://jaas.ai/docs/storage\n[juju-docs-actions]: https://jaas.ai/docs/actions\n[juju-docs-spaces]: https://jaas.ai/docs/spaces\n[juju-docs-config-apps]: https://juju.is/docs/configuring-applications\n[ceph-docs-removing-osds]: https://docs.ceph.com/docs/master/rados/operations/add-or-rm-osds/\n[ceph-docs-network-ref]: http://docs.ceph.com/docs/master/rados/configuration/network-config-ref\n[lp-bugs-charm-ceph-osd]: https://bugs.launchpad.net/charm-ceph-osd/+filebug\n[cdg-install-openstack]: https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/install-openstack.html\n[upstream-ceph-buckets]: https://docs.ceph.com/docs/master/rados/operations/crush-map/#types-and-buckets\n[upstream-ceph-bluestore]: https://docs.ceph.com/en/latest/rados/configuration/storage-devices/#bluestore\n[cloud-archive-ceph]: https://wiki.ubuntu.com/OpenStack/CloudArchive#Ceph_and_the_UCA\n[lp-bug-1858519]: https://bugs.launchpad.net/charm-ceph-osd/+bug/1858519\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenstack%2Fcharm-ceph-osd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenstack%2Fcharm-ceph-osd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenstack%2Fcharm-ceph-osd/lists"}