{"id":18077628,"url":"https://github.com/fukamachi/psychiq","last_synced_at":"2025-04-05T20:17:15.265Z","repository":{"id":136844413,"uuid":"46063607","full_name":"fukamachi/psychiq","owner":"fukamachi","description":"Background job processing for Common Lisp","archived":false,"fork":false,"pushed_at":"2024-04-22T01:33:35.000Z","size":169,"stargazers_count":54,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-02-11T19:13:09.665Z","etag":null,"topics":["common-lisp","message-queue"],"latest_commit_sha":null,"homepage":"","language":"Common Lisp","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fukamachi.png","metadata":{"files":{"readme":"README.markdown","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-11-12T15:55:02.000Z","updated_at":"2025-01-08T20:39:12.000Z","dependencies_parsed_at":"2024-01-27T09:06:38.276Z","dependency_job_id":"5cf253c2-62f5-44e1-a9a1-eb063f507905","html_url":"https://github.com/fukamachi/psychiq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fukamachi%2Fpsychiq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fukamachi%2Fpsychiq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fukamachi%2Fpsychiq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fukamachi%2Fpsychiq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fukamachi","download_url":"https://codeload.github.com/fukamachi/psychiq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393568,"owners_count":20931813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["common-lisp","message-queue"],"created_at":"2024-10-31T11:46:35.403Z","updated_at":"2025-04-05T20:17:15.244Z","avatar_url":"https://github.com/fukamachi.png","language":"Common Lisp","funding_links":[],"categories":["Interfaces to other package managers"],"sub_categories":["Third-party APIs"],"readme":"# Psychiq\n\n[![Build Status](https://travis-ci.org/fukamachi/psychiq.svg?branch=master)](https://travis-ci.org/fukamachi/psychiq)\n[![Coverage Status](https://coveralls.io/repos/fukamachi/psychiq/badge.svg?branch=master)](https://coveralls.io/r/fukamachi/psychiq)\n[![Quicklisp dist](http://quickdocs.org/badge/psychiq.svg)](http://quickdocs.org/psychiq/)\n\nPsychiq provides background job processing for Common Lisp applications inspired by Ruby's [Sidekiq](http://sidekiq.org).\n\n## Warning\n\nThis software is still ALPHA quality. The APIs will be likely to change.\n\n### Breaking changes\n\n1. **v0.2.0** - Removed namespaces.  See section below for details.\n\n\n## Getting Started\n\n### Writing a worker\n\n```common-lisp\n(psy:connect-toplevel :host \"localhost\" :port 6379)\n\n(defclass my-worker (psy:worker) ())\n(defmethod psy:perform ((worker my-worker) \u0026rest args)\n  ;; Do something\n  )\n```\n\nThe worker class is commonly written in an individual ASDF system because it must be shared with clients and servers.\n\n### Enqueueing (Client)\n\n```common-lisp\n;; Enqueueing to \"default\" queue\n(psy:enqueue 'my-worker '(\"arg1\" \"arg2\"))\n\n;; Enqueueing the job after 300 seconds.\n(psy:enqueue-in-sec 300 'my-worker '(\"arg1\" \"arg2\"))\n\n;; Enqueueing a large number of jobs at a time\n;; This is useful if you are pushing tens of thousands of jobs or more\n(psy:enqueue-bulk 'my-worker '(\"arg1\" \"arg2\") '(\"another\" \"one\") ...)\n```\n\nThe arguments must be simple JSON datatypes which can be serialized with Jonathan.\n\n### Starting processing (Server)\n\nPsychiq provides a [Roswell](https://github.com/snmsts/roswell) script for starting processing:\n\n```\n$ psychiq.ros --host localhost --port 6379 --system myapp-workers\n```\n\n```\n$ psychiq.ros -h\nUsage: psychiq.ros [option...]\n\nOptions:\n    -o, --host HOST                 Redis server host (default: localhost)\n    -p, --port PORT                 Redis server port (default: 6379)\n    -d, --db DBNUM                  Redis server db (default: 0)\n    -q, --queue QUEUE[,WEIGHT]      Queues to process with optional weights (several -q's allowed)\n    -c, --concurrency INT           Processor threads to use (default: 25)\n    -s, --system SYSTEM             ASDF system to load before starting (several -s's allowed)\n    -v, --verbose                   Print more verbose output\n    -n, --namespace NAMESPACE       Redis namespace (default: \"psychiq\")\n    -V, --version                   Print version\n    -h, --help                      Show help\n```\n\n## Worker options\n\n```common-lisp\n(defclass my-worker (psy:worker) ())\n\n;; Specify max retry attempts. (default: 25)\n(defmethod psy:worker-max-retries ((worker my-worker))\n  1000)\n\n;; Use a named queue to push. (default: \"default\")\n(defmethod psy:worker-queue-name ((worker my-worker))\n  \"my-worker-queue\")\n\n;; Disable jobs going to the dead job queue. (default: T)\n(defmethod psy:worker-use-dead-queue-p ((worker my-worker))\n  nil)\n\n;; Whether to save any error backtrace in the retry payload. (default: NIL)\n(defmethod psy:worker-use-backtrace-p ((worker my-worker))\n  t)\n```\n\n## Signals\n\n- INT: graceful shutdown, waits for all processors are idle.\n- TERM: shutdown immediately\n\n## Namespaces / Databases\n\nIn version 7 Sidekiq [removed namespaces](https://github.com/sidekiq/sidekiq/blob/main/docs/7.0-Upgrade.md#redis-namespace). We handle that by setting `psychiq.specials::*psychiq-namespace*` to `NIL`.  To revert back to the previous behaviour, simply set `*psychiq-namespace*` to `\"psychiq\"` (the previous default), or whatever namespace you want.\n\nThe recommended alternative to namespaces is to use different Redis databases.  This is handled with the `:db` keyword argument to `psy:connect-toplevel`, or the `--db` command line argument for the psychiq.ros script.\n\n## Error Handling\n\nWhen getting an error while performing a job, Psychiq will add the job to the \"retry\" queue. Jobs in the \"retry\" queue will be retried automatically with an exponential backoff. After 25 attempts, Psychiq move the job to the \"dead\" queue.\n\n## Web UI\n\nSince the data structure which Psychiq stores in Redis is compatible with Ruby's Resque/Sidekiq, Sidekiq's Web UI can be used.\n\nThe following assumes you are running Redis on localhost and you only want to access the dashboard from localhost.  This should be the least risky configuration if you are just going to cut/paste.\n\n```ruby\n# web.ru\n# Sidekiq dashboard\n#\n# Source: https://github.com/sidekiq/sidekiq/wiki/Monitoring#standalone\n#\n# Need to do the following:\n#   $ gem install rackup sidekiq securerandom rack rack-session\n#   $ rackup --host 127.0.0.1 ./web.ru\n\nrequire 'sidekiq'\n\nSidekiq.configure_client do |config|\n  config.redis = { :size =\u003e 1, url: 'redis://127.0.0.1:6379' }\nend\n\n\nrequire \"securerandom\"\nrequire \"rack/session\"\nrequire \"sidekiq/web\"\n\n# In a multi-process deployment, all Web UI instances should share\n# this secret key so they can all decode the encrypted browser cookies\n# and provide a working session.\n# Rails does this in /config/initializers/secret_token.rb\nsecret_key = SecureRandom.hex(32)\nuse Rack::Session::Cookie, secret: secret_key, same_site: true, max_age: 86400\nrun Sidekiq::Web\n```\n\n```\n$ gem install rackup sidekiq securerandom rack rack-session\n$ rackup --host 127.0.0.1 ./web.ru\n```\n\nIt will be up at http://127.0.0.1:9292 which allows you to see processes and can retry failed jobs manually.\n\n## Requirements\n\n* SBCL (compiled with sb-thread) or Clozure CL\n* Redis\n* [Roswell](https://github.com/snmsts/roswell) for command-line launcher script.\n\n## Installation\n\n```\n$ ros install psychiq\n```\n\n## Author\n\n* Eitaro Fukamachi (e.arrows@gmail.com)\n\n## Copyright\n\nCopyright (c) 2015-2017 Eitaro Fukamachi (e.arrows@gmail.com)\n\n## License\n\nLicensed under the LLGPL License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffukamachi%2Fpsychiq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffukamachi%2Fpsychiq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffukamachi%2Fpsychiq/lists"}