{"id":19398189,"url":"https://github.com/helloimalemur/reque","last_synced_at":"2025-06-10T10:39:40.848Z","repository":{"id":209969666,"uuid":"725387571","full_name":"helloimalemur/reQue","owner":"helloimalemur","description":"HTTP Request queue, replay, and prioritization micro-service. The name is a play on the word deque, request, and queue.","archived":false,"fork":false,"pushed_at":"2023-12-21T22:54:51.000Z","size":70,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-03T23:31:36.672Z","etag":null,"topics":["request-proxying","request-relay","rust","rust-lang","rust-rocket"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/helloimalemur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-11-30T03:07:51.000Z","updated_at":"2024-07-02T15:25:09.000Z","dependencies_parsed_at":"2023-12-13T04:09:08.288Z","dependency_job_id":null,"html_url":"https://github.com/helloimalemur/reQue","commit_stats":null,"previous_names":["helloimalemur/reque"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helloimalemur%2FreQue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helloimalemur%2FreQue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helloimalemur%2FreQue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helloimalemur%2FreQue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/helloimalemur","download_url":"https://codeload.github.com/helloimalemur/reQue/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helloimalemur%2FreQue/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259057221,"owners_count":22799111,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["request-proxying","request-relay","rust","rust-lang","rust-rocket"],"created_at":"2024-11-10T11:05:12.668Z","updated_at":"2025-06-10T10:39:40.817Z","avatar_url":"https://github.com/helloimalemur.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# reQue\nHTTP Request queue, replay, and prioritization micro-service.\nThe name is a play on the word deque, request, and queue.\n\n### For when your API can't say no\nQueue requests destined for slow backend servers when responding with 429 is unacceptable.\n\n### Intended Usage\nCreated to be used in a microservice based architecture, placed in-front of the slow-responding service.\nThink of it as a per-service-mini-proxy that doesn't drop your requests when drinking from a firehose.\n\n### Place reQue in front of your slow-to-consume / slow-to-respond backend servers to \"trickle\" incoming requests to them.\n```mermaid\nflowchart LR\n    A[External ASYNC API Request/Webhook] --\u003e|http/https| B(proxy)\n    B --\u003e C{reQue}\n    C --\u003e|/your/endpoint| D[This Slow Server]\n    C --\u003e|/your/other/endpoint?with=params| E[That Slow Server]\n    F[database]\u003c--\u003e|request queue| C{reQue}\n%%    C{reQue} --\u003e|request queue| F[database]\n```\n\n\n### Installation assumptions/requirements\n- [Rust](https://www.rust-lang.org/tools/install) is installed.\n- systemd based linux distribution.\n\n## Install\n```shell\nbash -e install.sh\n```\n\n## Bring your own SQL server\n    docker run -p 127.0.0.1:3306:3306  --name mdb -e MARIADB_ROOT_PASSWORD=Password123! -d mariadb:latest;\n    mariadb -h 127.0.0.1 -uroot -pPassword123! -e 'CREATE DATABASE reque;';\n    mariadb -D reque -h 127.0.0.1 -uroot -pPassword123! -e 'CREATE TABLE `requests` (`id` int(11) NOT NULL AUTO_INCREMENT,`method` varchar(255) NOT NULL,`host` varchar(255) NOT NULL,`port` varchar(255) NOT NULL,`uri` varchar(255) NOT NULL,`headers` varchar(255) NOT NULL,`body` varchar(6255) NOT NULL,PRIMARY KEY (`id`));';\n    mariadb -h 127.0.0.1 -uroot -pPassword123! -e \"CREATE USER 'dev'@'%' IDENTIFIED BY 'password';\";\n    mariadb -h 127.0.0.1 -uroot -pPassword123! -e \"GRANT ALL PRIVILEGES ON reque.* TO 'dev'@'%';\";\n    mariadb -h 127.0.0.1 -uroot -pPassword123! -e \"FLUSH PRIVILEGES;\";\n    \n\n## Create database\n```sql\nCREATE DATABASE reque;\n```\n\n## Create tables needed in the Database;\n\n```sql\nCREATE TABLE `requests` (`id` int(11) NOT NULL AUTO_INCREMENT,\n`method` varchar(255) NOT NULL,\n`host` varchar(255) NOT NULL,\n`port` varchar(255) NOT NULL,\n`uri` varchar(255) NOT NULL,\n`headers` varchar(255) NOT NULL,\n`body` varchar(6255) NOT NULL,\nPRIMARY KEY (`id`));\n```\n\n## Create database user\n```sql\nCREATE USER 'dev'@'%' IDENTIFIED WITH sha256_password BY 'password';\nCREATE USER 'dev'@'%' IDENTIFIED BY 'password';\nGRANT ALL PRIVILEGES ON reque.* TO 'dev'@'%';\nFLUSH PRIVILEGES;\n```\n\n## Edit config/Settings.toml\n```toml\n## general app, database, and external api settings\ndatabase_url = \"mysql://dev:password@localhost:3306/reque\"\ndatabase_name = \"reque\" ## name of queue database\napi_key = \"yourapikey\" ## for api-key protection - not currently enabled on any endpoints\nlog_path = \"log/requests.log\" ## logging is currently not working\nreque_service_port = \"8030\"\nremove_from_queue_on_failure = \"false\"\n## reque destination endpoint settings\nhttp_proto = \"http\" ## protocol of final destination server\nhttp_dest = \"localhost:7780\" ## final destination server\nreque_interval = \"3\" ## slow trickling requests to dest based on interval in seconds\nrequire_success = \"false\" ## receiving slow server must respond with 200\n```\n\n## test and dev;\n\n#### 1. Start reque\n```shell\ncargo run\n```\n\n#### 2. Start a test server to receive the funneled requests, such as [slow server](https://github.com/helloimalemur/Slow-Server) or \"./test_server.py [\u003cport\u003e] within this repo\"\n\n#### 3. Send requests into reque, which will be queued and funneled to the destination endpoint specified in Settings.toml\n```shell\n# create entry;\ncurl -X POST \"http://127.0.0.1:8030/plugins/shopify/\" -H \"Content-Type: application/json\" -d '{\"name\": \"John Doe\", \"age\": 30, \"city\": \"New York\"}'\n#create a slow-server test entry;\ncurl -X POST \"http://127.0.0.1:8030/delay/30/\" -H \"Content-Type: application/json\" -d '{\"name\": \"John Doe\", 30, \"city\": \"New York\"}'\n# create a lot of slow-server test entries;\nfor i in {00..500}; do curl -X POST \"http://127.0.0.1:8030/delay/3/\" -d \"$i\"; done;\n```\n#### 4. Observe requests being trickle funneled to the specified endpoint based on interval specified in Settings.toml\n\n[//]: # (```sql)\n[//]: # (INSERT INTO requests \u0026#40;method, host, port, uri, headers, body\u0026#41; VALUES \u0026#40;\"method\", \"host\", \"port\", \"uri\", \"headers\", \"body\"\u0026#41;;)\n[//]: # (```)\n\n\n## Development and Collaboration\n#### Feel free to open a pull request, please run the following prior to your submission please!\n    echo \"Run clippy\"; cargo clippy -- -D clippy::all\n    echo \"Format source code\"; cargo fmt -- --check\n\n\n\n### Resources\n    https://www.baeldung.com/cs/tokens-vs-sessions\n    https://api.rocket.rs/v0.4/rocket/http/enum.Cookies.html\n    https://api.rocket.rs/v0.4/rocket/request/trait.FromRequest.html\n    https://rocket.rs/v0.5-rc/guide/requests/#custom-guards\n    https://api.rocket.rs/v0.5-rc/rocket/request/trait.FromRequest.html\n    https://stackoverflow.com/questions/69377336/how-to-get-state-in-fromrequest-implementation-with-rocket\n    https://stackoverflow.com/questions/73868771/rust-rocket-with-sqlx-test-database-endpoints\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhelloimalemur%2Freque","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhelloimalemur%2Freque","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhelloimalemur%2Freque/lists"}