{"id":21973647,"url":"https://github.com/alash3al/exeq","last_synced_at":"2025-04-28T14:26:52.984Z","repository":{"id":72757411,"uuid":"405464676","full_name":"alash3al/exeq","owner":"alash3al","description":"painless task queue manager for shell commands with an intuitive cli interface (execute shell commands in distributed cloud-native queue manager).","archived":false,"fork":false,"pushed_at":"2021-09-24T22:14:35.000Z","size":69,"stargazers_count":14,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-30T10:33:41.164Z","etag":null,"topics":["background-jobs","cli","command-line","golang","queue","queue-workers","redis"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alash3al.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-11T19:20:50.000Z","updated_at":"2024-10-22T16:48:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"af94cbce-8826-40d2-9b16-6206c97e3e7f","html_url":"https://github.com/alash3al/exeq","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fexeq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fexeq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fexeq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alash3al%2Fexeq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alash3al","download_url":"https://codeload.github.com/alash3al/exeq/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251328655,"owners_count":21571958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["background-jobs","cli","command-line","golang","queue","queue-workers","redis"],"created_at":"2024-11-29T15:33:51.711Z","updated_at":"2025-04-28T14:26:52.957Z","avatar_url":"https://github.com/alash3al.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"EXEQ\n======\n\u003e **DOCS STILL IN PROGRESS**.  \n\u003e Execute shell commands in queues via cli or http interface.\n\nFeatures\n========\n- Simple intuitive tiny cli app.\n- Modular queue backends (currently it supports redis as backend but we should support more in the future like `sqs`, `kafka`, `postgres`, ...).\n- Powerful configurations thanks to HCL by hashicrop.\n- OpenMetrics `/metrics` endpoint to inspect the queue via promethues.\n- Error reporting via sentry and stdout/stderr logging.\n- Easily create shortcuts for repeated shell commands (we name it Macros).\n- Limit the job execution time.\n\nComponents\n==========\n- the configuration file which uses hcl as a configuration language.\n- the binary itself which you can get from the [releases page](https://github.com/alash3al/exeq/releases).\n\n#### Config\n```hcl\n// here we define the logging related configs.\n// kindly note that this config file is preprocessed at first with environment expander\n// this means that you can easily use any ENV var in any value here i.e: ${HOME}\nlogging {\n    // available level names are: \"disable\" \"fatal\" \"error\" \"warn\" \"info\" \"debug\".\n    // NOTE: the value is case sensitive.\n    log_level = \"debug\"\n\n    // sentry dsn (for error reporting).\n    // see https://sentry.io/\n    // you can do this: sentry_dsn = ${SENTRY_DSN}, but you have to export that env\n    // before running exeq.\n    sentry_dsn = \"\"\n}\n\n// http server related configs.\nhttp {\n    // the address to start listening on.\n    listen = \":1215\"\n\n    // whether to enable/disable access logs.\n    access_logs = true\n}\n\n// queue related configs.\nqueue {\n    // the driver used as queue backend.\n    // the available drivers are: [rmq].\n    driver = \"rmq\"\n\n    // the data source name or the connection string for the selected driver.\n    dsn = \"redis://localhost:6379/1\"\n\n    // queue workers count.\n    workers_count = 5\n\n    // the duration between each poll from the queue\n    // a duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, \n    // such as \"300ms\", \"-1.5h\" or \"2h45m\". Valid time units are \"ns\", \"us\" (or \"µs\"), \"ms\", \"s\", \"m\", \"h\".\n    poll_duration = \"1s\"\n\n    // how many times a should we retry a failed job?\n    retry_attempts = 3\n\n    // each driver needs to keep jobs history, but you may not have to keep it for ever\n    // so this history block helps you to configure the history feature.\n    history {\n        // for how long should we keep our history?\n        // a duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, \n        // such as \"300ms\", \"-1.5h\" or \"2h45m\". Valid time units are \"ns\", \"us\" (or \"µs\"), \"ms\", \"s\", \"m\", \"h\".\n        retention_period = \"48h\"\n    }\n}\n\n// here we define a macro which is considered an alias for a command.\n// the macro name is \"example1\".\nmacro \"example1\" {\n    // the command you want to execute when you call this macro.\n    // you can pass arguments to the command and start using it in the command\n    // you have to take a look at this first: https://pkg.go.dev/text/template\n    // the main variable you can access is `{{.Args}}` which is a map of key=\u003evalue.\n    // here we cat the data from hello.txt which we mounted before, then echo the value from\n    // the arguments map exists in a key called message (just like $args['message']).\n    command = \"cat ./hello.txt \u0026\u0026 echo {{.Args.message}}\"\n\n    // after how long time should we kill the job?\n    // a duration string is a possibly signed sequence of decimal numbers, each with optional fraction and a unit suffix, \n    // such as \"300ms\", \"-1.5h\" or \"2h45m\". Valid time units are \"ns\", \"us\" (or \"µs\"), \"ms\", \"s\", \"m\", \"h\".\n    // empty means \"no time limit\".\n    max_execution_time = \"\"\n\n    // sometimes when you run this service within a cloud engine like k8s or docker container\n    // you will have to define multiple configmaps/volumes mounts,\n    // to simplify this job, we can only mount exeq configurations file and add any other file required by the job\n    // into the following mounts block.\n    mount \"./hello.txt\" {\n        content = \"hello world\"\n    }\n}\n```\n\n#### Exeq Binary\n\u003e exeq is an intuitive cli app, you can just write `exeq help` from your shell and go with its help.\n\u003e the binary consists of the following subcommands\n```shell\n   queue:work     start the queue worker(s)\n   enqueue:macro  submit macro to the queue\n   enqueue:cmd    submit a raw shell command to the queue\n   queue:jobs     list the jobs\n   queue:stats    show queue stats\n   serve:http     start the http server which enables you to enqueue macros and inspect the queue via /metrics endpoint with the help of promethues\n   help, h        Shows a list of commands or help for one command\n```\n\n#### Steps\n- Run a queue daemon via `exeq queue:work`\n- Optional run the http server as per your needs.\n- Start submitting commands either via:\n    - HTTP API `POST /enqueue/{MACRO_NAME}`, this endpoint accepts a json message which will be passed to the underlying shell command as args,\n    - CLI\n        - `exeq enqueue:cmd echo hello world`\n        - `exeq enqueue:macro MACRO_NAME -a k=v -a k2=v2`\n- You may want to see the queue stats via:\n    - CLI:\n        - `exeq queue:stats`\n        - `watch exeq queue:stats`\n    - HTTP API:\n        - `GET /`\n        - `GET /metrics` (a promethues metrics endpoint)\n- You may want to list jobs history `exeq queue:jobs`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falash3al%2Fexeq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falash3al%2Fexeq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falash3al%2Fexeq/lists"}