{"id":24487148,"url":"https://github.com/probably-not/flame-ec2","last_synced_at":"2025-04-13T21:06:36.168Z","repository":{"id":271645377,"uuid":"905338563","full_name":"probably-not/flame-ec2","owner":"probably-not","description":"A FLAME backend for EC2 Machines","archived":false,"fork":false,"pushed_at":"2025-03-21T14:11:18.000Z","size":139,"stargazers_count":34,"open_issues_count":5,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-13T21:06:25.829Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/probably-not.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-18T16:14:37.000Z","updated_at":"2025-03-23T17:26:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"03aaae0d-91df-42a7-be28-42abd4cdd8ba","html_url":"https://github.com/probably-not/flame-ec2","commit_stats":null,"previous_names":["probably-not/flame-ec2"],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/probably-not%2Fflame-ec2","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/probably-not%2Fflame-ec2/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/probably-not%2Fflame-ec2/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/probably-not%2Fflame-ec2/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/probably-not","download_url":"https://codeload.github.com/probably-not/flame-ec2/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248782270,"owners_count":21160717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-21T15:33:32.896Z","updated_at":"2025-04-13T21:06:36.159Z","avatar_url":"https://github.com/probably-not.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FlameEC2\n\nA FLAME backend for EC2 Machines.\n\n## Why not K8S/ECS/Docker/etc?\n\nI don't like complicating my deployments too much - while I have used K8S and ECS and Docker Compose and have spent years learning their ins and outs, I've recently started preferring simple bare-metal style deployments. Just give me a VPS with SSH access, plus something like Ansible, CodeDeploy (I am using AWS after all), or just simple rsync and a systemd service and I'm happy.\n\nWith that in mind - I want my FLAME cluster to shine (see what I did there?) without needing all this fluff as well.\n\n## Setup\n\nSee the moduledoc for `FlameEC2` for extensive documentation on different options.\n\nThe main thing that you need to set up properly is your `s3_bundle_url` value. This defines the bundle that will be loaded onto the runner machine, in order to run the application. There are various strategies to set up the release bundle url - for example, you may always push the latest release to a specific directory in S3 (the \"latest\" directory) and simply hardcode pointing to this URL. Alternatively, you may want to maintain versioining and use a Git tag or Git commit hash to point to the correct bundle - the version can be set in the build when compiling your release so that you can know which version to point to on any given release.\n\n## Usage Recommendation\n\n### FLAME.call Timeouts and FLAME.Pool Boot Timeouts\n\nEC2 Machines can take some time coming up (even sometimes several minutes) due to how AWS manages capacity in their availability zones. The time is typically under 2 minutes, although it is highly dependent on the instance types that you are trying to raise. Because of this, the `:boot_timeout` for your `FLAME.Pool` and the `:timeout` option for `FLAME.call` should be set high enough to avoid timeouts happening due to waiting for an instance to move from Pending to Running.\n\n### Warm Pool\n\nIf you want to ensure that you don't cause all of your function calls to take a long time due to waiting for capacity, it's highly recommended to have a warm pool (i.e. a pool with a minimum greater than 0). This will ensure that for initial calls, you will not wait for an instance to become ready. **Note:** While having a warm pool does ensure that you have an instance at the beginning, if your scale grows, you will eventually have to wait for new instances to boot up.\n\n### Idle Runner Timeouts\n\nIn a similar vein to the above two recommendations - setting the `:idle_shutdown_after` option on your configured `FLAME.Pool` can help in ensuring that you do not scale down too quickly. The default value is set to `30_000` (30 seconds), but setting this to a higher value can help ensure that you aren't constantly raising and terminating instances.\n\n### Example Configuration\n\nBased on the above recommendations, an example configuration that may work well would look something like this:\n\n```elixir\n{\n  FLAME.Pool,\n  name: FlameEC2Demo.Pool,\n  # Optionally, set the minimum to 1 so that we can run 100 concurrent tasks without worrying about waiting.\n  min: 0,\n  max: 10,\n  max_concurrency: 100,\n  # Set a global FLAME.call timeout that is higher than the boot timeout to ensure that we do not timeout calls before booting\n  timeout: 180_000,\n  # Set the boot timeout to a high enough number that we can have enough time to raise an instance\n  boot_timeout: 150_000,\n  # Set the idle shutdown timeout to high enough that we don't constantly raise and terminate instances\n  idle_shutdown_after: 60_000,\n}\n```\n\n## Debugging Issues\n\nThe `FlameEC2` backend works by setting up an EC2 UserData script which will initialize the runner instance with a systemd service that is running the release bundle that you specify. If the runner is not able to start, there may be an issue during the start script which causes the whole process to fail. If there is a problem, you can see the logs of the start script by using the systemd journal as shown here:\n\n```sh\nsudo journalctl -t flame_ec2_init\n```\n\nThis should output any logs related to the start script to show whether or not there is a problem.\n\n## Installation\n\nIf [available in Hex](https://hex.pm/docs/publish), the package can be installed\nby adding `flame_ec2` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:flame_ec2, \"~\u003e 1.0.0\"}\n  ]\nend\n```\n\nDocumentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc)\nand published on [HexDocs](https://hexdocs.pm). Once published, the docs can\nbe found at \u003chttps://hexdocs.pm/flame_ec2\u003e.\n\n## Contributing\n\nFeel free to fork and make PRs! I'm definitely happy to have eyes on this and to get feedback so we can take care of edge cases.\n\nFor contributions, ensure that you run `git update-index --skip-worktree compose-env/ec2_mock_config.json`. The mock config gets reset\nand resynced in every `iex` session to ensure that the local compose environment is up to date for localized tests. This means that\nit is constantly changing, and therefore we should not track its changes. However, I do want to ensure that the file is distributed with\nthe repository, so that developers don't have to know to create it or have to do any setup before getting started.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprobably-not%2Fflame-ec2","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprobably-not%2Fflame-ec2","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprobably-not%2Fflame-ec2/lists"}