{"id":13424424,"url":"https://github.com/GrottoPress/mel","last_synced_at":"2025-03-15T18:34:58.966Z","repository":{"id":52544000,"uuid":"351918690","full_name":"GrottoPress/mel","owner":"GrottoPress","description":"A scalable asynchronous event-driven jobs engine","archived":false,"fork":false,"pushed_at":"2024-08-13T16:40:54.000Z","size":354,"stargazers_count":37,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-13T02:22:38.735Z","etag":null,"topics":["background-jobs","cron-jobs","crystal","instant-jobs","message-queue","periodic-jobs","redis","task-scheduler"],"latest_commit_sha":null,"homepage":"","language":"Crystal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GrottoPress.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-26T21:41:29.000Z","updated_at":"2024-11-24T15:01:24.000Z","dependencies_parsed_at":"2024-04-16T17:36:54.220Z","dependency_job_id":"9afe1046-2585-41a9-8c29-e96fbcf94a2b","html_url":"https://github.com/GrottoPress/mel","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GrottoPress%2Fmel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GrottoPress%2Fmel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GrottoPress%2Fmel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GrottoPress%2Fmel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GrottoPress","download_url":"https://codeload.github.com/GrottoPress/mel/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243775845,"owners_count":20346275,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-jobs","cron-jobs","crystal","instant-jobs","message-queue","periodic-jobs","redis","task-scheduler"],"created_at":"2024-07-31T00:00:54.179Z","updated_at":"2025-03-15T18:34:58.955Z","avatar_url":"https://github.com/GrottoPress.png","language":"Crystal","funding_links":[],"categories":["Crystal"],"sub_categories":[],"readme":"# Mel\n\n**Mel** is an asychronous event-driven jobs processing engine designed to scale. *Mel* simplifies jobs management by abstracting away the nuances of scheduling and running jobs.\n\nIn *Mel*, a scheduled job is called a *task*. A single job may be scheduled in multiple ways, yielding multiple tasks from the same job.\n\nMel schedules all tasks in the chosen storage backend as a set of task `id`s sorted by their times of next run. For recurring tasks, the next run is scheduled right after the current run completes.\n\nThis makes the storage backend the *source of truth* for schedules, allowing to easily scale out *Mel* to multiple instances (called *workers*), or replace or stop workers without losing schedules.\n\n*Mel* supports *bulk scheduling* of jobs as a single atomic unit. There's also support for *sequential scheduling* to track a series of jobs and perform some action after they are all complete.\n\n### Types of tasks\n\n1. **Instant Tasks:** These are tasks that run only once after they are scheduled, either immediately or at some specified time in the future.\n\n1. **Periodic Tasks:** These are tasks that run regularly at a specified interval. They may run forever, or till some specified time in the future.\n\n1. **Cron Tasks:** These are tasks that run according to a specified schedule in *Unix Cron* format. They may run forever, or till some specified time in the future.\n\n## Installation\n\n1. Add the dependency to your `shard.yml`:\n\n   ```yaml\n   dependencies:\n     mel:\n       github: GrottoPress/mel\n     #redis: # Uncomment if using the Redis backend\n     #  github: jgaskins/redis\n   ```\n\n1. Run `shards update`\n\n1. Require and configure *Mel* in your app (we'll configure workers later):\n\n   ```crystal\n   # -\u003e\u003e\u003e src/app/config.cr\n\n   # ...\n\n   require \"mel\"\n\n   require \"../jobs/**\"\n\n   Mel.configure do |settings|\n     settings.error_handler = -\u003e(error : Exception) { puts error.message }\n     settings.timezone = Time::Location.load(\"Africa/Accra\")\n   end\n\n   Log.setup(Mel.log.source, :info, Log::IOBackend.new)\n   # Redis::Connection::LOG.level = :info # Uncomment if using the Redis backend\n\n   # ...\n   ```\n\n   - Using the Redis backend\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/config.cr\n\n     # ...\n\n     require \"mel/redis\"\n\n     Mel.configure do |settings|\n       # ...\n       settings.store = Mel::Redis.new(\n         \"redis://localhost:6379/0\",\n         namespace: \"mel\"\n       )\n       # ...\n     end\n\n     # ...\n     ```\n\n   - Using the Memory backend (Not for production use)\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/config.cr\n\n     # ...\n\n     require \"mel\"\n\n     Mel.configure do |settings|\n       # ...\n       settings.store = Mel::Memory.new\n       # ...\n     end\n\n     # ...\n     ```\n\n   - Skip storage\n\n     You may disable storage altogether by setting `Mel.settings.store` to `nil` (This is the default).\n\n## Usage\n\n1. Define job:\n\n   ```crystal\n   # -\u003e\u003e\u003e src/jobs/do_some_work.cr\n\n   struct DoSomeWork\n     include Mel::Job # \u003c= Required\n\n     def initialize(@arg_1 : Int32, @arg_2 : String)\n     end\n     # \u003c= Instance vars must be JSON-serializable\n\n     # (Required)\n     #\n     # Main operation to be performed.\n     # Called in a new fiber.\n     def run\n       # \u003c\u003c Do work here \u003e\u003e\n     end\n\n     # Called in the main fiber, before spawning the fiber\n     # that calls the `#run` method above.\n     def before_run\n       # ...\n     end\n\n     # Called in the same fiber that calls `#run`.\n     # `success` is `true` only if the run succeeded.\n     def after_run(success)\n       if success\n         # ...\n       else\n         # ...\n       end\n     end\n\n     # Called in the main fiber before enqueueing the task in\n     # the store.\n     def before_enqueue\n       # ...\n     end\n\n     # Called in the main fiber after enqueueing the task in\n     # the store. `success` is `true` only if the enqueue succeeded.\n     def after_enqueue(success)\n       if success\n         # ...\n       else\n         # ...\n       end\n     end\n   end\n   ```\n\n1. Schedule job:\n\n   - Run job now:\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/some_file.cr\n\n     DoSomeWork.run(arg_1: 5, arg_2: \"value\")\n     # \u003c= Alias: DoSomeWork.run_now(...)\n     ```\n\n   - Run job after given delay:\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/some_file.cr\n\n     DoSomeWork.run_in(5.minutes, arg_1: 5, arg_2: \"value\")\n     ```\n\n     The given `Time::Span` can be negative. Eg: `DoSomeWork.run_in(-5.minutes, ...)`. This may be useful for prioritizing certain tasks.\n\n   - Run job at specific time:\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/some_file.cr\n\n     DoSomeWork.run_at(10.minutes.from_now, arg_1: 5, arg_2: \"value\")\n     ```\n\n     The specified `Time` can be in the past. Eg: `DoSomeWork.run_at(-10.minutes.from_now, ...)`. This may be useful for prioritizing certain tasks.\n\n   - Run periodically:\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/some_file.cr\n\n     DoSomeWork.run_every(10.minutes, for: 1.hour, arg_1: 5, arg_2: \"value\")\n     ```\n\n     This will do the first run 10 minutes from now. If you would like to do the first run some other time, specify that in a `from:` argument:\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/some_file.cr\n\n     DoSomeWork.run_every(10.minutes, from: Time.local, for: 1.hour, arg_1: 5, arg_2: \"value\")\n     ```\n\n     Instead of `for:`, you may use `till:` and specify a `Time`. Leave those out to run forever.\n\n   - Run on a Cron schedule:\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/some_file.cr\n\n     DoSomeWork.run_on(\"0 */2 * * *\", for: 6.hours, arg_1: 5, arg_2: \"value\")\n     ```\n\n     This will do the first run relative to now. For instance, if the time now is 03:00, the first run would be at 04:00, the next run at 06:00, and so on. If you would like to do the first run relative to some other time, specify that in a `from:` argument:\n\n     ```crystal\n     # -\u003e\u003e\u003e src/app/some_file.cr\n\n     DoSomeWork.run_on(\"0 */2 * * *\", from: 3.days.from_now, for: 6.hours, arg_1: 5, arg_2: \"value\")\n     ```\n\n     Instead of `for:`, you may use `till:` and specify a `Time`. Leave those out to run forever.\n\n   The `DoSomeWork.run_*` methods accept the following additional arguments:\n\n   - `retries`: Number of times to attempt a task after it fails, before giving up. This could be specified as a simple integer (eg: `3`), or a list of backoffs (eg: `{2, 4, 1}`, or `{2.seconds, 4.seconds, 1.second}`). Default: `{2, 4, 8, 16}`. A task fails when any exception is raised during run.\n\n1. Start *Mel*:\n\n   - As its own process (compiled separately):\n\n     ```crystal\n     # -\u003e\u003e\u003e src/worker.cr\n\n     require \"mel\"\n\n     require \"./app/**\"\n\n     Mel.configure do |settings|\n       settings.batch_size = -100\n       settings.poll_interval = 3.seconds\n       settings.worker_id = ENV[\"WORKER_ID\"].to_i\n     end\n\n     Mel.start\n     # \u003c= Blocks forever, polls for due tasks and runs them.\n     # \u003c= You may stop Mel by sending `Signal::INT` or `Signal::TERM`.\n     # \u003c= Mel will wait for all running tasks to complete before exiting.\n     ```\n\n   - As part of your app (useful for testing):\n\n     ```crystal\n     # -\u003e\u003e\u003e spec/spec_helper.cr\n\n     # ...\n\n     require \"mel/spec\"\n\n     Mel.configure do |settings|\n       settings.batch_size = -1\n       settings.poll_interval = 1.millisecond\n       settings.worker_id = 1\n     end\n\n     Spec.before_each { Mel::Task::Query.truncate }\n\n     Spec.after_suite do\n       Mel.stop\n       Mel::Task::Query.truncate\n     end\n     # \u003c= `Mel.stop` waits for all running tasks to complete before exiting\n\n     Mel.start_async\n\n     # ...\n     ```\n\n1. Configure compile targets:\n\n   ```yaml\n   # -\u003e\u003e\u003e shard.yml\n\n   # ...\n\n   targets:\n     app:\n       main: src/app.cr\n     worker:\n       main: src/worker.cr\n\n   # ...\n   ```\n\n### Job templates\n\nA job's `.run_*` methods allow scheduling that single job in multiple ways. However, there may be situations where you need to schedule a job the same way, every time.\n\n*Mel* comes with `Mel::Job::Now`, `Mel::Job::In`, `Mel::Job::At`, `Mel::Job::Every` and `Mel::Job::On` templates to do exactly this:\n\n```crystal\n# Define job\nstruct DoSomeWorkNow\n  include Mel::Job::Now # \u003c= Required\n\n  def initialize(@arg_1 : Int32, @arg_2 : String)\n  end\n\n  # (Required)\n  def run\n    # \u003c\u003c Do work here \u003e\u003e\n  end\nend\n\n# Schedule job\nDoSomeWorkNow.run(arg_1: 5, arg_2: \"value\")\n# \u003c= Alias: `DoSomeWorkNow.run_now(...)`\n```\n\n```crystal\n# Define job\nstruct DoSomeWorkIn\n  include Mel::Job::In # \u003c= Required\n\n  def initialize(@arg_1 : Int32, @arg_2 : String)\n  end\n\n  # (Required)\n  def run\n    # \u003c\u003c Do work here \u003e\u003e\n  end\nend\n\n# Schedule job\nDoSomeWorkIn.run_in(10.minutes, arg_1: 5, arg_2: \"value\")\n```\n\n```crystal\n# Define job\nstruct DoSomeWorkAt\n  include Mel::Job::At # \u003c= Required\n\n  def initialize(@arg_1 : Int32, @arg_2 : String)\n  end\n\n  # (Required)\n  def run\n    # \u003c\u003c Do work here \u003e\u003e\n  end\nend\n\n# Schedule job\nDoSomeWorkAt.run_at(Time.local(2021, 6, 9, 5), arg_1: 5, arg_2: \"value\")\n```\n\n```crystal\n# Define job\nstruct DoSomeWorkEvery\n  include Mel::Job::Every # \u003c= Required\n\n  def initialize(@arg_1 : Int32, @arg_2 : String)\n  end\n\n  # (Required)\n  def run\n    # \u003c\u003c Do work here \u003e\u003e\n  end\nend\n\n# Schedule job\nDoSomeWorkEvery.run_every(2.hours, arg_1: 5, arg_2: \"value\")\n# \u003c= Overload: `.run_every 2.hours, for: 5.hours`\n# \u003c= Overload: `.run_every 2.hours, till: 9.hours.from_now`\n```\n\n```crystal\n# Define job\nstruct DoSomeWorkOn\n  include Mel::Job::On # \u003c= Required\n\n  def initialize(@arg_1 : Int32, @arg_2 : String)\n  end\n\n  # (Required)\n  def run\n    # \u003c\u003c Do work here \u003e\u003e\n  end\nend\n\n# Schedule job\nDoSomeWorkOn.run_on(\"0 8 1 * *\", arg_1: 5, arg_2: \"value\")\n# \u003c= Overload: `.run_on \"0 8 1 * *\", for: 100.weeks`\n# \u003c= Overload: `.run_on \"0 8 1 * *\", till: Time.local(2099, 12, 31)`\n```\n\nA template excludes all methods not relevant to that template. For instance, calling `.run_every` or `.run_now` for a `Mel::Job::At` template won't compile.\n\nAll other methods and callbacks usable in a regular job may be used in a template, including `before_*` and `after_*` callbacks.\n\nYou may `include` more than one template in a single job. For instance, including `Mel::Job::At` and `Mel::Job::Every` in a job means you can call `.run_at` and `.run_every` methods for that job.\n\nAdditionally, *Mel* comes with two grouped templates: `Mel::Job::Instant` and `Mel::Job::Recurring`.\n\n`Mel::Job::Instant` is equivalent to `Mel::Job::Now`, `Mel::Job::In` and `Mel::Job::At` combined. `Mel::Job::Recurring` is the equivalent of `Mel::Job::Every` and `Mel::Job::On` combined.\n\n`Mel::Job` is itself a grouped template that combines all the other templates.\n\n### Specifying task IDs\n\nYou may specify an ID whenever you schedule a new job, thus: `DoSomeWork.run_*(... id: \"1001\", ...)`. If not specified, *Mel* automatically generates a unique **dynamic** ID for the task.\n\nDynamic task IDs may be OK for *triggered* jobs (jobs triggered by some kind of user interaction), such as a job that sends an email notification whenever a user logs in.\n\nHowever, there may be jobs that are scheduled unconditionally when your app starts (*global* jobs). For example, sending invoices at the beginning of every month. You should specify unique **static** IDs for such tasks.\n\nOtherwise, every time the app (re)starts, jobs are scheduled again, each time with a different set of IDs. The store would accept the new schedules because the IDs are different, resulting in duplicated scheduling of the same jobs.\n\nThis is particularly important if you run multiple instances of your app. Hardcoding IDs for *global* jobs means that all instances hold the same IDs, so cannot reschedule a job that has already been scheduled by another instance.\n\nA task ID may be a mixture of static and dynamic parts. For instance, you may include the current month and year for a global job that runs once a month, to ensure it is never scheduled twice within the same month.\n\n### Bulk scheduling\n\nA common pattern is to break up long-running tasks into smaller tasks. For example:\n\n```crystal\nstruct SendAllEmails\n  include Mel::Job\n\n  def initialize(@users : Array(User))\n  end\n\n  def run\n    @users.each { |user| send_email(user) }\n  end\n\n  private def send_email(user)\n    # Send email\n  end\nend\n\n# Schedule job\nusers = # ...\nSendAllEmails.run(users: users)\n```\n\nThe above job would run in a single fiber, managed by whichever worker pulls this task at run time. This could mean too much work for a single worker if the number of users is sufficiently large.\n\nMoreover, some mails may be sent multiple times if the task is retried as a result of failure. Ideally, jobs should be idempotent, and as atomic as possible.\n\nThe preferred approach is to define a job that sends email to one user, and schedule that job for as many users as needed:\n\n```crystal\nstruct SendAllEmails\n  include Mel::Job\n\n  def initialize(@users : Array(User))\n  end\n\n  def run\n    return if @users.empty?\n\n    # Pushes all jobs atomically, at the end of the block.\n    #\n    transaction do |store|\n      # Pass `store` to `.run_*`.\n      @users.each { |user| SendEmail.run(store: store, user: user) }\n    end\n  end\n\n  struct SendEmail\n    include Mel::Job\n\n    def initialize(@user : User)\n    end\n\n    def run\n      send_email(@user)\n    end\n\n    private def send_email(user)\n      # Send email\n    end\n  end\nend\n\n# Schedule job\nusers = # ...\nSendAllEmails.run(users: users)\n# \u003c= Any `.run_*` method could be called here, as with any job.\n```\n\n### Sequential scheduling\n\nBulk scheduling works OK as a *fire-and-forget* mechanism. However, you may need to keep track of a series of jobs as a single unit, and perform some action only after the last job is done.\n\nThis is where sequential scheduling comes in handy. *Mel*'s event-driven design allows chaining jobs, by scheduling the next after the current one completes:\n\n```crystal\nstruct SendAllEmails\n  include Mel::Job\n\n  def initialize(@users : Array(User))\n  end\n\n  def run\n    @users[0]?.try do |user|\n      send_email(user) # \u003c= Send first email\n    end\n  end\n\n  def after_run(success)\n    return unless success\n\n    if @users[1]?\n      self.class.run(users: @users[1..]) # \u003c= Schedule next email\n    else # \u003c= All emails have been sent\n      # Do something\n    end\n  end\n\n  private def send_email(user)\n    # Send email\n  end\nend\n\n# Schedule job\nusers = # ...\nSendAllEmails.run(users: users)\n```\n\nAlthough the example above involves a single job, sequential scheduling can be applied to multiple different jobs, each representing a step in a workflow, with each job scheduling the next job in its `#after_run` callback:\n\n```crystal\nstruct SomeJob\n  include Mel::Job\n  \n  def run\n    # Do something\n  end\n\n  def after_run(success)\n    SomeStep.run if success\n  end\n\n  struct SomeStep\n    include Mel::Job\n\n    def run\n      # Do something\n    end\n\n    def after_run(success)\n      SomeOtherStep.run if success\n    end\n  end\n\n  struct SomeOtherStep\n    include Mel::Job\n\n    def run\n      # Do something\n    end\n\n    def after_run(success)\n      # All done; do something\n    end\n  end\nend\n```\n\n### Tracking progress\n\n*Mel* provides a progress tracker for jobs. This is particularly useful for tracking multiple jobs representing a series of steps in a workflow:\n\n```crystal\n# -\u003e\u003e\u003e src/app/config.cr\n\n# ...\n\nMel.configure do |settings|\n  settings.progress_expiry = 1.day\nend\n\n# ...\n```\n\n```crystal\n# -\u003e\u003e\u003e src/jobs/some_job.cr\n\nstruct SomeJob\n  include Mel::Job\n\n  def initialize\n    @progress = Mel::Progress.start(id: \"some_job\", description: \"Awesome job\")\n  end\n\n  # ...\n\n  def after_run(success)\n    return @progress.fail unless success\n\n    transaction do |store|\n      SomeStep.run(store: store, progress: @progress)\n      @progress.move(50, store) # \u003c= Move to 50%\n    end\n  end\n\n  struct SomeStep\n    include Mel::Job::Now\n\n    def initialize(@progress : Mel::Progress)\n    end\n\n    # ...\n\n    def after_run(success)\n      return @progress.fail unless success\n\n      transaction do |store|\n        SomeOtherStep.run(store: store, progress: @progress)\n        @progress.move(80, store) # \u003c= Move to 80%\n      end\n    end\n  end\n\n  struct SomeOtherStep\n    include Mel::Job::Now\n\n    def initialize(@progress : Mel::Progress)\n    end\n\n    # ...\n\n    def after_run(success)\n      return @progress.fail unless success\n      @progress.succeed # \u003c= Move to 100%\n    end\n  end\nend\n\n# Schedule job\nSomeJob.run\n\n# Track progress\n#\n# This may, for instance, be used in a route in a web application.\n# Client-side javascipt can query this route periodically, and\n# show response using a progress tracker UI.\n#\nreport = Mel::Progress.track(\"some_job\")\n\nreport.try do |_report|\n  _report.description\n  _report.id\n  _report.value\n\n  _report.failure?\n  _report.running?\n  _report.success?\n\n  _report.started?\n  _report.ended?\nend\n```\n\nYou may delete progress data in specs thus:\n\n```crystal\n# -\u003e\u003e\u003e spec/spec_helper.cr\n\n# ...\n\nSpec.before_each do\n  # ...\n  Mel::Progress::Query.truncate\n  # ...\nend\n\nSpec.after_suite do\n  # ...\n  Mel::Progress::Query.truncate\n  # ...\nend\n\n# ...\n```\n\n### Jobs *security*\n\nA *Mel* worker waits for all running tasks to complete before exiting, if it received a `Signal::INT` or a `Signal::TERM`, or if you called `Mel.stop` somewhere in your code. This means jobs are never lost mid-flight.\n\nJobs are not lost even if there is a force shutdown of the worker process, since *Mel* does not delete a task from the store until it is complete. The worker can pick off where it left off when it comes back online.\n\n*Mel* relies on the `worker_id` setting to achieve this. Each worker, therefore, must set a *unique*, *static* integer ID, so it knows which *pending* tasks it owns.\n\nOnce a task enters the *pending* state, only the worker that put it in that state can run it. So if you need to take down a worker permanently, ensure that it completes all pending tasks by sending the appropriate signal.\n\n### Scaling out\n\nBecause each worker requires it's own unique `.worker_id`, autoscaling as used in classic distributed architectures should not be used, since auto-scaled replicas would inherit the same configuration as the original instance.\n\nThis would lead to multiple workers using the same `.worker_id`, which could result in pending jobs being run multiple times; once each for each replica that starts up.\n\nInstead, it is recommended that a new service be registered for each worker that is to be deployed, and the appropriate `.worker_id` set for each.\n\n- Using `Procfile`:\n\n  ```procfile\n  # -\u003e\u003e Procfile\n\n  # ...\n  worker_1: export WORKER_ID=1 \u0026\u0026 ./bin/worker\n  worker_2: export WORKER_ID=2 \u0026\u0026 ./bin/worker\n  worker_3: export WORKER_ID=3 \u0026\u0026 ./bin/worker\n  # ...\n  ```\n\n- Using docker compose for swarm:\n\n  ```yaml\n  # -\u003e\u003e docker-compose.yml\n\n  # ...\n  services:\n    worker_1:\n      command: ./bin/worker\n      environment:\n        WORKER_ID: \"1\"\n      deploy:\n        replicas: 1\n    worker_2:\n      command: ./bin/worker\n      environment:\n        WORKER_ID: \"2\"\n      deploy:\n        replicas: 1\n    worker_3:\n      command: ./bin/worker\n      environment:\n        WORKER_ID: \"3\"\n      deploy:\n        replicas: 1\n  # ...\n  ```\n\nAnother option is to accept the worker ID as a command argument:\n\n```crystal\n# -\u003e\u003e src/worker.cr\n\n# ...\nARGV.first?.try { |worker_id| Mel.settings.worker_id = worker_id.to_i }\n\nMel.start\n```\n\n- Using `Procfile`:\n\n  ```procfile\n  # -\u003e\u003e Procfile\n\n  # ...\n  worker_1: ./bin/worker 1\n  worker_2: ./bin/worker 2\n  worker_3: ./bin/worker 3\n  # ...\n  ```\n\n- Using docker compose for swarm:\n\n  ```yaml\n  # -\u003e\u003e docker-compose.yml\n\n  # ...\n  services:\n    worker_1:\n      command: ./bin/worker 1\n      deploy:\n        replicas: 1\n    worker_2:\n      command: ./bin/worker 2\n      deploy:\n        replicas: 1\n    worker_3:\n      command: ./bin/worker 3\n      deploy:\n        replicas: 1\n  # ...\n  ```\n\n- Using config for [Fly.io](https://fly.io):\n\n  ```toml\n  # -\u003e\u003e fly.toml\n\n  # ...\n  [processes]\n    worker_1 = './bin/worker 1'\n    worker_2 = './bin/worker 2'\n    worker_3 = './bin/worker 3'\n  # ...\n  ```\n\n  Ensure no spare machines are created by passing `--ha=false` to `fly deploy` command.\n\n### Smart polling\n\n*Mel*'s `batch_size` setting allow setting a limit on the number of due tasks to retrieve and run each poll, and, consequently, the number of fibers spawned to handle those tasks.\n\nIf the setting is a positive integer `N`, *Mel* would pull and run `N` due tasks each poll.\n\nIf it is a negative integer `-N`  (other than `-1`), the number of due tasks pulled and ran each poll would vary such that the total number of running tasks would not be greater than `N`.\n\n`-1` sets *no* limits. *Mel* would pull as many tasks as are due each poll, and run all of them.\n\n## Integrations\n\n### *Carbon* mailer\n\n\u003csmall\u003eLink: https://github.com/luckyframework/carbon\u003c/small\u003e\n\n1. Require `mel/carbon`, after your emails:\n\n   ```crystal\n   # -\u003e\u003e\u003e src/app.cr\n\n   # ...\n   require \"emails/base_email\"\n   require \"emails/**\"\n\n   require \"mel/carbon\"\n   # ...\n   ```\n\n1. Set up base email:\n\n   ```crystal\n   # -\u003e\u003e\u003e src/emails/base_email.cr\n\n   abstract class BaseEmail \u003c Carbon::Email\n     # ...\n     include JSON::Serializable\n     # ...\n   end\n   ```\n\n1. Configure deliver later strategy:\n\n   ```crystal\n   # -\u003e\u003e\u003e config/email.cr\n\n   BaseEmail.configure do |settings|\n     # ...\n     settings.deliver_later_strategy = Mel::Carbon::DeliverLater.new\n     # ...\n   end\n   ```\n\n## Development\n\nCreate a `.env.sh` file:\n\n```bash\n#!/bin/bash\n\nexport REDIS_URL='redis://localhost:6379/0'\n```\n\nUpdate the file with your own details. Then run tests with `source .env.sh \u0026\u0026 crystal spec -Dpreview_mt`.\n\n## Contributing\n\n1. [Fork it](https://github.com/GrottoPress/mel/fork)\n1. Switch to the `master` branch: `git checkout master`\n1. Create your feature branch: `git checkout -b my-new-feature`\n1. Make your changes, updating changelog and documentation as appropriate.\n1. Commit your changes: `git commit`\n1. Push to the branch: `git push origin my-new-feature`\n1. Submit a new *Pull Request* against the `GrottoPress:master` branch.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGrottoPress%2Fmel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FGrottoPress%2Fmel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FGrottoPress%2Fmel/lists"}