{"id":19954087,"url":"https://github.com/Eventual-Inc/daft-cli","last_synced_at":"2025-05-03T19:32:04.034Z","repository":{"id":257813075,"uuid":"851791527","full_name":"Eventual-Inc/daft-launcher","owner":"Eventual-Inc","description":"A simple launcher for spinning up and managing Ray clusters for the Daft Query Engine.","archived":false,"fork":false,"pushed_at":"2024-10-24T00:32:47.000Z","size":507,"stargazers_count":6,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-24T14:42:18.007Z","etag":null,"topics":["daft","distributed-computing","query","ray"],"latest_commit_sha":null,"homepage":"https://eventual-inc.github.io/daft-launcher","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Eventual-Inc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-03T17:54:21.000Z","updated_at":"2024-10-23T03:54:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"73d76cd4-8c2b-4fe9-8b3c-f2ab159561e7","html_url":"https://github.com/Eventual-Inc/daft-launcher","commit_stats":null,"previous_names":["eventual-inc/daft-launcher"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eventual-Inc%2Fdaft-launcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eventual-Inc%2Fdaft-launcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eventual-Inc%2Fdaft-launcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eventual-Inc%2Fdaft-launcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Eventual-Inc","download_url":"https://codeload.github.com/Eventual-Inc/daft-launcher/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224370826,"owners_count":17300101,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["daft","distributed-computing","query","ray"],"created_at":"2024-11-13T01:18:52.723Z","updated_at":"2025-05-03T19:31:58.800Z","avatar_url":"https://github.com/Eventual-Inc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://emojis.wiki/thumbs/emojis/rocket.webp\" alt=\"Daft Launcher\"\u003e\n\u003c/div\u003e\n\n\u003cbr\u003e\n\n[![PyPI Package](https://github.com/Eventual-Inc/daft-launcher/actions/workflows/publish-to-pypi.yaml/badge.svg)](https://github.com/Eventual-Inc/daft-launcher/actions/workflows/publish-to-pypi.yaml)\n[![Deploy mdBook](https://github.com/Eventual-Inc/daft-launcher/actions/workflows/deploy-mdbook.yaml/badge.svg)](https://github.com/Eventual-Inc/daft-launcher/actions/workflows/deploy-mdbook.yaml)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)\n[![Latest](https://img.shields.io/github/v/tag/Eventual-Inc/daft-launcher?label=latest\u0026logo=GitHub)](https://github.com/Eventual-Inc/daft-launcher/tags)\n[![License](https://img.shields.io/badge/daft_launcher-docs-red.svg)](https://eventual-inc.github.io/daft-launcher)\n\n# Daft Launcher CLI Tool\n\n`daft-launcher` is a simple launcher for spinning up and managing Ray clusters for [`daft`](https://github.com/Eventual-Inc/Daft).\n\nFor a deeper introduction, please refer to our [documentation](https://eventual-inc.github.io/daft-launcher/).\n\n## Goal\n\nGetting started with Daft in a local environment is easy.\nHowever, getting started with Daft in a cloud environment is substantially more difficult.\nSo much more difficult, in fact, that users end up spending more time setting up their environment than actually playing with our query engine.\n\nDaft Launcher aims to solve this problem by providing a simple CLI tool to remove all of this unnecessary heavy-lifting.\n\n## Capabilities\n\nWhat Daft Launcher is capable of:\n1. Spinning up clusters.\n2. Listing all available clusters (as well as their statuses).\n3. Submitting jobs to a cluster.\n4. Connecting to the cluster (to view the Ray dashboard and submit jobs using the Ray protocol).\n5. Spinning down clusters.\n6. Creating configuration files.\n7. Running raw SQL statements using Daft's SQL API.\n\n## Currently supported cloud providers\n\n- [x] AWS\n- [ ] GCP\n- [ ] Azure\n\n## Usage\n\n### Pre-requisites\n\nYou'll need some python package manager installed.\nWe recommend using `uv` for all things python.\n\n#### AWS\n\nIf you're using AWS, you'll need:\n1. A valid AWS account with the necessary IAM role to spin up EC2 instances.\n  This IAM role can either be created by you (assuming you have the appropriate permissions).\n  Or this IAM role will need to be created by your administrator.\n2. The [AWS CLI](https://aws.amazon.com/cli/) installed and configured on your machine.\n3. To login using the AWS CLI.\n  For full instructions, please look [here](https://google.com).\n\n### Installation\n\nUsing `uv`:\n\n```bash\n# create project\nmkdir my-project\ncd my-project\n\n# initialize project and setup virtual env\nuv init\nuv venv\nsource .venv/bin/activate\n\n# install launcher\nuv pip install daft-launcher\n```\n\n### Example\n\nAll interactions with Daft Launcher are primarily communicated via a configuration file.\nBy default, Daft Launcher will look inside your `$CWD` for a file named `.daft.toml`.\nYou can override this behaviour by specifying a custom configuration file.\n\n```bash\n# create a new configuration file\n# will create a file named `.daft.toml` in the current working directory\ndaft init-config\n# or optionally, pass in a custom name\ndaft init-config my-custom-config.toml\n\n# spin up a cluster\ndaft up\n# or optionally, pass in a custom config file\ndaft up -c my-custom-config.toml\n\n# list all the active clusters (can have multiple clusters running at the same time)\ndaft list\n\n# submit a directory and a command to run on the cluster\ndaft submit --working-dir \u003c...\u003e -- command arg1 arg2 ...\n# or optionally, pass in a custom config file\ndaft submit -c my-custom-config.toml --working-dir $WORKING_DIR -- command arg1 arg2 ...\n\n# run a direct SQL query against the daft query engine running in the remote cluster\ndaft sql -- \"SELECT * FROM my_table WHERE column = 'value'\"\n# or optionally, pass in a custom config file\ndaft sql -c my-custom-config.toml -- \"SELECT * FROM my_table WHERE column = 'value'\"\n\n# spin down a cluster\ndaft down\n# or optionally, pass in a custom name\ndaft down -c my-custom-config.toml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEventual-Inc%2Fdaft-cli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FEventual-Inc%2Fdaft-cli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FEventual-Inc%2Fdaft-cli/lists"}