{"id":18085008,"url":"https://github.com/coderofsalvation/bullmq","last_synced_at":"2025-06-13T15:12:46.814Z","repository":{"id":16615113,"uuid":"19370003","full_name":"coderofsalvation/bullmq","owner":"coderofsalvation","description":"*deprecated* simple api management rate limiting: http proxy + throttler + message/job queue based on nodejs bull","archived":false,"fork":false,"pushed_at":"2020-09-04T22:45:18.000Z","size":182,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-06T00:13:47.855Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coderofsalvation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"custom":"https://gumroad.com/l/hGYGh"}},"created_at":"2014-05-02T08:36:42.000Z","updated_at":"2020-05-28T18:44:35.000Z","dependencies_parsed_at":"2022-08-27T15:09:30.967Z","dependency_job_id":null,"html_url":"https://github.com/coderofsalvation/bullmq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/coderofsalvation/bullmq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coderofsalvation%2Fbullmq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coderofsalvation%2Fbullmq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coderofsalvation%2Fbullmq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coderofsalvation%2Fbullmq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coderofsalvation","download_url":"https://codeload.github.com/coderofsalvation/bullmq/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coderofsalvation%2Fbullmq/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259668726,"owners_count":22893136,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-31T15:09:02.937Z","updated_at":"2025-06-13T15:12:46.790Z","avatar_url":"https://github.com/coderofsalvation.png","language":"JavaScript","funding_links":["https://gumroad.com/l/hGYGh"],"categories":[],"sub_categories":[],"readme":"bullmq\n======\n\u003cimg alt=\"\" src=\"doc/bullmq.png\"/\u003e\n\na httpproxythrottlingmessagejobqueuedomainrouter.\n\n### What is it?\n\nIts a enormous pillow to put in front of your webserver.\nPrevent server/application-panic by limiting and rerouting webrequests to different areas/servers.\n\n### Isnt there already software for this?\n\nThere are many enterprise-ish packages out there, but I wanted something deadsimple to configure.\nNext to that AMQP-support (not finished) will be able to scale out to big servicebusses like RabbitMQ e.g.\n\n### Concept\n\nHere's a sequencediagram (because sequencediagrams are cool :) hope it makes sense though)\n\n\u003cimg src=\"doc/seqdiagram1.png\"/\u003e\n\n### A simple configuration\n\nLets have a look at the conf.d/myproject.js file:\n\n    exports.proxy = function(config){  \n      config[\"localhost:8080\"]        = [\"http://127.0.0.1:80\"];\n      config[\"mydomain.org:8080\"]     = [\"http://127.0.0.1:80\"];\n      config[\"www.mydomain.org:8080\"] = [\"http://127.0.0.1:80\"];\n      config[\"ws.mydomain.org:8080\"]  = [\"http://127.0.0.1:8111\"];\n      return config;\n    }\n\nAbove are domainredirects: localhost on 8080 will redirect to port 80. Other domains can \nbe entered (for example ws. for websocket traffic).\nMultiple arrayvalues will result in random round-robin behaviour (quickndirty scalable).\n    \n    exports.limiters = function(config){\n      config[\"localhost:8080\"]     = { unit: \"minute\", rate:90 };  // 90 requests per minute \n      config[\"localhost:8080/foo\"] = { unit: \"minute\", rate:20 };  // overrides 90 with 20 \n      return config;\n    }\n\nAbove you can limit per domain and/or url. The units are: day,hour,minute,second.\nThis is handy to ensure your projects will not overload your server.\nFor example, one could setup 3 subdomains for api-calls: \n\n* `freemium.api.com` \n* `basic.api.com` \n* `premium.api.com`\n* \nJust set different rates for these subdomains, and bullmq will reply to clients like so when rate are exceeded:\n\n    {succes:false, code: 429, msg:\"Too Many Requests - your IP is being rate limited\",data:{}}\n\nAnd now for workers:\n\n    exports.queue = function(config){  \n      config[\"localhost:8080\"]    = {\n        \"/foo\": { \n          queue: \"/request_web\",       // put '/foo' in queue 'request_web'\n          timeout: 500                 // generate timeout if no redis subscribers respond within 500 milliseconds\n        }      \n      };\n      return config;\n    }\n   \nHere you can queue a request for later processing using nodejs bull (b2b apis/messaging/long running tasks).\nIf a redis- or amqp-subscriber does snot respond with a 'completed'-event to the task, bullmq will reply:\n\n    {succes:false, code: 104, msg:\"server could not handle your request in time\",data:{}}\n\nIf a subscriber *does* respond within the timeout-timeframe, it can respond with a \"queued\", \"processing\" or \"ok\" reply.\nExample: user wants to encode 4 video's, clients are firing hundreds of push requests to the databases etc.\n\n\n### Syslog\n\nlogging can be directed to stdout and/or syslog.\nHere's some syslog output :\n\n    May  9 10:01:41 bullmq [26639]: ACCEPT =\u003e www.yourdomain/tpl/front/gfx/search_black.png from 94.214.179.213 (bucket:232.06\n    May  9 10:01:41 bullmq [26639]: ACCEPT =\u003e www.yourdomain/lib/flop/site/tpl/gfx/terminal.empty.gif from 94.214.179.213 (buc\n    May  9 10:01:45 bullmq [26639]: ACCEPT =\u003e www.yourdomain/js/?hash=MjAxMS0wNi90dHlyZWNvcmQtMTMwODMxODg4Mw== from 65.49.68.1\n    May  9 10:01:45 bullmq [26639]: ACCEPT =\u003e www.yourdomain/js/?hash=MjAxMS0wNi90dHlyZWNvcmQtMTMwNjk2NjA2NQ== from 65.49.2.17\n    May  9 10:01:48 bullmq [26639]: ACCEPT =\u003e www.yourdomain/lib/flop/js/gfx/flop.button.white.png from 65.49.68.165 (bucket:2\n    May  9 10:01:48 bullmq [26639]: ACCEPT =\u003e www.yourdomain/lib/flop/js/gfx/flop.skin.gif from 65.49.2.175 (bucket:227.572666\n    May  9 10:01:50 bullmq [26639]: ACCEPT =\u003e www.yourdomain/r/MjAxMy0xMC90dHlyZWNvcmR0dHktMTM4MTU3NjE5Nnw4MHgyNA== from 157.5\n    May  9 10:02:10 bullmq [26639]: ACCEPT =\u003e www.yourdomain/ from 94.214.179.213 (bucket:227.02533333333335)\n    May  9 10:02:11 bullmq [26639]: ACCEPT =\u003e www.yourdomain/lib/core/functions.js from 94.214.179.213 (bucket:226.07126666666\n\nNOTE: the bucket number is how much requests are possible for that url before rate-limiting kicks in.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoderofsalvation%2Fbullmq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoderofsalvation%2Fbullmq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoderofsalvation%2Fbullmq/lists"}