{"id":18633718,"url":"https://github.com/scylladb/scylla-machine-image","last_synced_at":"2025-07-19T05:04:28.511Z","repository":{"id":36988563,"uuid":"222907516","full_name":"scylladb/scylla-machine-image","owner":"scylladb","description":null,"archived":false,"fork":false,"pushed_at":"2024-05-22T11:09:21.000Z","size":537,"stargazers_count":17,"open_issues_count":18,"forks_count":25,"subscribers_count":14,"default_branch":"next","last_synced_at":"2024-05-22T12:28:23.074Z","etag":null,"topics":["aws","scylladb"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scylladb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-11-20T10:06:31.000Z","updated_at":"2024-05-22T12:28:30.904Z","dependencies_parsed_at":"2023-02-19T05:16:15.211Z","dependency_job_id":"c47ce71c-ddc7-4adb-9dd7-8c3a390871b8","html_url":"https://github.com/scylladb/scylla-machine-image","commit_stats":null,"previous_names":[],"tags_count":187,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-machine-image","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-machine-image/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-machine-image/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scylladb%2Fscylla-machine-image/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scylladb","download_url":"https://codeload.github.com/scylladb/scylla-machine-image/tar.gz/refs/heads/next","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248358865,"owners_count":21090446,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","scylladb"],"created_at":"2024-11-07T05:16:02.590Z","updated_at":"2025-04-11T07:32:29.928Z","avatar_url":"https://github.com/scylladb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scylla Machine Image\nProvides following\n- Create an image with pre-installed Scylla\n- Allow to configure the database when an instance is launched first time\n- Easy cluster creation\n\n## OS Package\nRPM/DEB package that is pre-installed in the image.\nResponsible for configuring Scylla during first boot of the instance.\n\n## Create an image\n### AWS\n```shell script\naws/ami/build_ami.sh\n```\n\n## Scylla AMI user-data Format v2\n\nScylla AMI user-data should be passed as a json object, as described below\n\nsee AWS docs for how to pass user-data into ec2 instances:\n[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-add-user-data.html](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-add-user-data.html)\n---\n### EC2 User-Data\nUser Data that can pass when create EC2 instances\n\n* **Object Properties**\n    * **scylla_yaml** ([`Scylla YAML`](#scylla_yaml)) – Mapping of all fields that would pass down to scylla.yaml configuration file\n    * **scylla_startup_args** (*list*) – embedded information about the user that created the issue (NOT YET IMPLEMENTED) (*default=’[]’*)\n    * **developer_mode** ([*boolean*](https://docs.python.org/library/stdtypes.html#boolean-values)) – Enables developer mode (*default=’false’*)\n    * **post_configuration_script** ([*string*](https://docs.python.org/library/stdtypes.html#str)) – A script to run once AMI first configuration is finished, can be a string encoded in base64. (*default=’’*)\n    * **post_configuration_script_timeout** ([*int*](https://docs.python.org/library/stdtypes.html#int)) – Time in seconds to limit the post_configuration_script (*default=’600’*)\n    * **start_scylla_on_first_boot** ([*boolean*](https://docs.python.org/library/stdtypes.html#boolean-values)) – If true, scylla-server would boot at AMI boot (*default=’true’*)\n\n### \u003ca href=\"scylla_yaml\"\u003e\u003c/a\u003eScylla YAML\nAll fields that would pass down to scylla.yaml configuration file\n\nsee [https://docs.scylladb.com/operating-scylla/scylla-yaml/](https://docs.scylladb.com/operating-scylla/scylla-yaml/) for all the possible configuration availble\nlisted here only the one get defaults scylla AMI\n\n* **Object Properties**    \n    * **cluster_name** ([*string*](https://docs.python.org/library/stdtypes.html#str)) – Name of the cluster (*default=`generated name that would work for only one node cluster`*)\n    * **auto_bootstrap** ([*boolean*](https://docs.python.org/library/stdtypes.html#boolean-values)) – Enable auto bootstrap (*default=’true’*)\n    * **listen_address** ([*string*](https://docs.python.org/library/stdtypes.html#str)) – Defaults to ec2 instance private ip\n    * **broadcast_rpc_address** ([*string*](https://docs.python.org/library/stdtypes.html#str)) – Defaults to ec2 instance private ip\n    * **endpoint_snitch** ([*string*](https://docs.python.org/library/stdtypes.html#str)) – Defaults to ‘org.apache.cassandra.locator.Ec2Snitch’\n    * **rpc_address** ([*string*](https://docs.python.org/library/stdtypes.html#str)) – Defaults to ‘0.0.0.0’\n    * **seed_provider** (*mapping*) – Defaults to ec2 instance private ip\n\n### Example usage of user-data\n\nSpinning a new node connecting to “10.0.219.209” as a seed, and installing cloud-init-cfn package at first boot.\n\n#### using json\n```json\n{\n     \"scylla_yaml\": {\n         \"cluster_name\": \"test-cluster\",\n         \"experimental\": true,\n         \"seed_provider\": [{\"class_name\": \"org.apache.cassandra.locator.SimpleSeedProvider\",\n                            \"parameters\": [{\"seeds\": \"10.0.219.209\"}]}],\n     },\n     \"post_configuration_script\": \"#! /bin/bash\\nyum install cloud-init-cfn\",\n     \"start_scylla_on_first_boot\": true\n}\n```\n\n#### using yaml\n```yaml\nscylla_yaml:\n  cluster_name: test-cluster\n  experimental: true\n  seed_provider:\n    - class_name: org.apache.cassandra.locator.SimpleSeedProvider\n      parameters:\n        - seeds: 10.0.219.209\npost_configuration_script: \"#! /bin/bash\\nyum install cloud-init-cfn\"\nstart_scylla_on_first_boot: true\n```\n\n#### using mimemultipart\n\nIf other feature of cloud-init are needed, one can use mimemultipart, and pass\na json/yaml with `x-scylla/yaml` or `x-scylla/json`\n\nmore information on cloud-init multipart user-data:\n\nhttps://cloudinit.readthedocs.io/en/latest/topics/format.html#mime-multi-part-archive\n\n```mime\nContent-Type: multipart/mixed; boundary=\"===============5438789820677534874==\"\nMIME-Version: 1.0\n\n--===============5438789820677534874==\nContent-Type: x-scylla/yaml\nMIME-Version: 1.0\nContent-Disposition: attachment; filename=\"scylla_machine_image.yaml\"\n\nscylla_yaml:\n  cluster_name: test-cluster\n  experimental: true\n  seed_provider:\n    - class_name: org.apache.cassandra.locator.SimpleSeedProvider\n      parameters:\n        - seeds: 10.0.219.209\npost_configuration_script: \"#! /bin/bash\\nyum install cloud-init-cfn\"\nstart_scylla_on_first_boot: true\n\n--===============5438789820677534874==\nContent-Type: text/cloud-config; charset=\"us-ascii\"\nMIME-Version: 1.0\nContent-Transfer-Encoding: 7bit\nContent-Disposition: attachment; filename=\"cloud-config.txt\"\n\n#cloud-config\ncloud_final_modules:\n- [scripts-user, always]\n\n--===============5438789820677534874==--\n```\n\nexample of creating the multipart message by python code:\n\n```python\nimport json\nfrom email.mime.base import MIMEBase\nfrom email.mime.multipart import MIMEMultipart\n\nmsg = MIMEMultipart()\n\nscylla_image_configuration = dict(\n    scylla_yaml=dict(\n        cluster_name=\"test_cluster\",\n        listen_address=\"10.23.20.1\",\n        broadcast_rpc_address=\"10.23.20.1\",\n        seed_provider=[{\n            \"class_name\": \"org.apache.cassandra.locator.SimpleSeedProvider\",\n            \"parameters\": [{\"seeds\": \"10.23.20.1\"}]}],\n    )\n)\npart = MIMEBase('x-scylla', 'json')\npart.set_payload(json.dumps(scylla_image_configuration, indent=4, sort_keys=True))\npart.add_header('Content-Disposition', 'attachment; filename=\"scylla_machine_image.json\"')\nmsg.attach(part)\n\ncloud_config = \"\"\"\n#cloud-config\ncloud_final_modules:\n- [scripts-user, always]\n\"\"\"\npart = MIMEBase('text', 'cloud-config')\npart.set_payload(cloud_config)\npart.add_header('Content-Disposition', 'attachment; filename=\"cloud-config.txt\"')\nmsg.attach(part)\n\nprint(msg)\n```\n\n## Creating a Scylla cluster using the Machine Image\n### AWS - CloudFormation\nUse template `aws/cloudformation/scylla.yaml`.\nCurrently, maximum 10 nodes cluster is supported.\n\n## Building scylla-machine-image package\n\n### RedHat like - RPM\n\nCurrently the only supported mode is:\n\n```\ndist/redhat/build_rpm.sh --target centos7 --cloud-provider aws\n```\n\nBuild using Docker\n\n```\ndocker run -it -v $PWD:/scylla-machine-image -w /scylla-machine-image  --rm centos:7.2.1511 bash -c './dist/redhat/build_rpm.sh -t centos7 -c aws'\n```\n\n### Ubuntu - DEB\n\n```\ndist/debian/build_deb.sh\n```\n\nBuild using Docker\n\n```\ndocker run -it -v $PWD:/scylla-machine-image -w /scylla-machine-image  --rm ubuntu:20.04 bash -c './dist/debian/build_deb.sh'\n```\n\n## Building docs\n\n```bash\npython3 -m venv .venv\nsource .venv/bin/activate\npip install sphinx sphinx-jsondomain sphinx-markdown-builder\nmake html\nmake markdown\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscylladb%2Fscylla-machine-image","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscylladb%2Fscylla-machine-image","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscylladb%2Fscylla-machine-image/lists"}