{"id":32136977,"url":"https://github.com/circonus-labs/fq","last_synced_at":"2025-10-21T04:54:48.252Z","repository":{"id":3834267,"uuid":"4915553","full_name":"circonus-labs/fq","owner":"circonus-labs","description":"F@#$*\u0026%Q (Message queue that is fast, brokered, in C and gets out of your way)","archived":false,"fork":false,"pushed_at":"2025-05-23T19:36:00.000Z","size":1886,"stargazers_count":286,"open_issues_count":2,"forks_count":31,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-10-05T10:30:10.100Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"epoberezkin/ajv","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/circonus-labs.png","metadata":{"files":{"readme":"README.md","changelog":"ChangeLog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-07-05T21:27:23.000Z","updated_at":"2025-09-28T06:18:30.000Z","dependencies_parsed_at":"2024-12-16T19:31:32.976Z","dependency_job_id":"dfc23745-caa3-4304-b45a-8ed1c103b34b","html_url":"https://github.com/circonus-labs/fq","commit_stats":{"total_commits":387,"total_committers":22,"mean_commits":17.59090909090909,"dds":0.5038759689922481,"last_synced_commit":"9b63252867414a746d5502df17d927ca2e8ac2a7"},"previous_names":[],"tags_count":40,"template":false,"template_full_name":null,"purl":"pkg:github/circonus-labs/fq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circonus-labs%2Ffq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circonus-labs%2Ffq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circonus-labs%2Ffq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circonus-labs%2Ffq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/circonus-labs","download_url":"https://codeload.github.com/circonus-labs/fq/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circonus-labs%2Ffq/sbom","scorecard":{"id":283151,"data":{"date":"2025-08-11","repo":{"name":"github.com/circonus-labs/fq","commit":"aa2bb0149df754edd6fa82e8f30ee161eaaa2954"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.2,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":5,"reason":"Found 14/27 approved changesets -- score normalized to 5","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 17 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-17T16:28:29.735Z","repository_id":3834267,"created_at":"2025-08-17T16:28:29.736Z","updated_at":"2025-08-17T16:28:29.736Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280207203,"owners_count":26290616,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2025-10-21T04:54:46.374Z","updated_at":"2025-10-21T04:54:48.245Z","avatar_url":"https://github.com/circonus-labs.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fq.\n\n\u003ca href=\"https://scan.coverity.com/projects/circonus-labs-fq\"\u003e\n  \u003cimg alt=\"Coverity Scan Build Status\"\n       src=\"https://scan.coverity.com/projects/13357/badge.svg\"/\u003e\n\u003c/a\u003e\n\nfq is a *brokered* message queue using a publish subscribe model.  It is architected for performance and isn't (today) designed for large numbers of connected clients.\n\n\n    +------------+                        +-----------+\n    |- exchange -|\u003c-- (msg publication) --|- client0 -|\n    +------------+                        +-----------+\n    |- routemap -|\n    +------------+\n        |      |              +---------+\n        |      +--------------|- queue -|\n        |                     +---------+\n    +---------+                  |\n    |- queue -|                  |   +-----------+\n    +---------+                  +---|- client1 -|\n            |                        +-----------+\n            |  +-----------+\n            +--|- client2 -|\n            |  +-----------+\n            |\n       +-----------+\n       |- client3 -|\n       +-----------+\n\n## Terminology\n\n### Broker\n\nThe `fqd` process. The daemon through which all knowledge passes.\n\n### Peers\n\nPeers are connected `fqd` processes.  It is important to note that peers are unidirectional.  If A peers with B, then A will act as a client to B. If you want bidirectional peering, you must specify that A peers with B and B peers with A.  The system aims to prevent cyclic delivery of messages efficiently.\n\nAdding peers is done directly via fqd's sqlite DB store:\n\n```\n; sqlite3 /var/lib/fq/fqd.sqlite\nsqlite\u003e INSERT INTO \"upstream\"\n              (host, port, source, password, exchange, program, permanent_binding)\n        VALUES('peerB',8765,'fqd-peera//mem:drop,private,backlog=4096','none','logging','prefix:\"http.access.json.\"','false');\n```\n\n### Client\n\n * [C client - libfq](https://github.com/postwait/fq/blob/master/fq.h#L164-L205)\n * [Java client - fq.jar](https://github.com/postwait/fq/blob/master/java/src/com/omniti/labs/FqClientImplInterface.java)\n * [Node.js client - fq](https://www.npmjs.com/package/fq)\n * [Go client - fq](https://godoc.org/github.com/postwait/gofq)\n * submission-only /submit API (see below)\n\nA client is an applications connection to fq over TCP/IP to send or receive messages. A client makes two TCP/IP connections to fq.  An application can present itself to fq as multiple clients at one time (by opening new pairs of connections). See Queues for reasons why.\n\n### Exchanges\n\nExchanges are like buses on which messages may be sent.  You cannot send a message without doing so on an exchange.  Exchanges are created within fq on-demand.\n\n### Queues\n\nQueues are queues. If you stick something in one end, you should expect it to come out the other.  A single queue may have multiple clients subscribed.  When a client connects, it is attached to one and only one queue.  If an application wishes to attach to more than one queue, it should present as multiple clients.  Queues use a competitive consumption model meaning that if multiple clients are attached to a single queue, the messages sent to that queue will be distributed over the clients such that no two clients will see the same message.\n\n#### Queue Types\n\nQueues can be of type `mem` or `disk`.  The contents of memory queues will not survive restarts.\n\nVarious parameters can be set on a queue using the syntax `type:param1,param2`.\n\n#### Sharing\n\nQueues with the `public` parameter can have multiple clients connected to them (in which case they compete for messages).  If you want a private queue you can specify the `private` parameter.\n\n#### Policy\n\nQueues can either have a `block` or `drop` policy.  The drop policy means that messages that would be routed to a queue that is full will be dropped and never delivered.  The block policy will cause the publisher to wait until there is room in the queue.  The block policy makes no sense on a disk queue.\n\n#### Backlog\n\nThe `backlog=\u003cnumber\u003e` parameter will specify how many messages may be held in the queue before the block or drop policies are applied.\n\n#### Permanence\n\nIf you want a queue to be remembered by fqd, you can specify `permanent` as a flag.  If you'd like for fqd to forget the queue after all clients have disconnected, you can specify the `transient` flag.  If neither flag is specified, then an existing queue will retain its previous permanence setting or a new transient queue will be created.\n\n#### Examples:\n\nA queue called `bob` will be in memory, allowed to have multiple clients connected to it, with a drop policy and an allowable message backlog of 100000 messages: `bob/mem:public,drop,backlog=100000`\n\nA connection client will specify username/queue.  A user \"USER\" connecting to the aforementioned queue would connect as `USER/bob/mem:public,drop,backlog=100000`\n\n### Messages\n\nMessages are, of course, a payload and metadata.\n\n#### Message metadata\n\nSome are set by the broker.\n \n * sender [set by the broker]\n * hops (a list of fqd via which the message passed)\n\nOthers are set by the sender. \n\n * exchange (up to 127 bytes)\n * route (up to 127 bytes)\n * id (128 bits). The first 64 bits the sender shall control, the latter 64bits the broker *might* control.\n\n### Routes and Programs\n\nRoutes and programs define how messages sent on exchanges are placed in queues:\n\n- A receiver that connects to an fq-broker specifies a program that filters the messages on the exchange.\n- A sender specifies a route for every message as part of the metadata\n\nPrograms follow the following syntax (cf. `fqd.h`):\n\n```\nPROGRAM: \u003cprefix|exact\u003e:string RULES*\nRULE: (RULE)\nRULE: (RULE \u0026\u0026 RULE)\nRULE: (RULE || RULE)\nRULE: EXPR\nEXPR: function(args)\nargs: arg\nargs: arg, args\narg: \"string\"\narg: true|false\narg: [0-9][0-9]*(?:.[0-9]*)\n\nfunctions are dynamically loadable with type signature\nstrings: s, booleans: b, integers: d\nfunction: substr_eq(9.3,10,\"tailorings\",true)\nC symbol: fqd_route_prog__substr_eq__ddsb(int nargs, valnode_t *args);\n ```\n\nIn particular:\n\n- Every program starts with either `prefix:` or `exact:`\n- The program `prefix:` matches all rules\n- The program string is matched against the message route\n\nThe following rule functions are defined in `fq_prog.c`:\n\n- `fqd_route_prog__sample__d()` -- subsample the stream\n- `fqd_route_prog__route_contains__s()` -- check if route contains a string\n- `fqd_route_prog__payload_prefix__s()` -- check if payload starts with prefix\n- `fqd_route_prog__payload_contains__s()` -- check if payload contains a string\n- `fqd_route_prog__true__()` -- always true\n\nExamples:\n\n- `prefix:` -- matches all messages\n- `prefix:bla` or `prefix:\"bla\"` -- matches all messages with rules starting with the sting 'bla'\n- `prefix: payload_prefix(\"M\")` -- matches messages where the payload starts with 'M'\n- `prefix:foo (payload_prefix(\"M\") \u0026\u0026 route_contains(\"bar\"))` -- matches messages where the payload starts with 'M' and route starts with \"foo\" and moreover contains \"bar\"\n\n## Protocol\n\nInformation on command and message protocol is found in `docs/fq_protocol.md`\n\n### HTTP superposition\n\nThe Fq protocol also acts as a non-compliant HTTP server (though compliant enough of most clients and browsers).  Fq ships with a web UI that allows inspecting real-time state and performance.\n\n#### GET /stats.json\n\nexposes current exchange, queue, and client information.\n\n#### POST /submit\n\nAn endpoint allowing message submission without a full and stateful Fq connection.  It expects the following headers:\n\n * ```X-Fq-User```,\n * ```X-Fq-Route```, and\n * ```X-Fq-Exchange```.\n \n The HTTP client *MUST* provide a Content-Length header corresponding to the payload content (no chunked submission).  The payload is treated as the raw message box without any special encoding.\n\nExample:\n\n```\ncurl -X POST -H \"X-Fq-User: user\" -H 'X-Fq-Route: bla' -H 'X-Fq-Exchange: test' localhost:8765/submit --data \"TEST\"\n```\n\n## Building\n\nRequirements:\n* C compiler\n* GNU make\n* libuuid\n* sqlite3\n* [jlog](https://github.com/omniti-labs/jlog)\n* [libbcd](https://github.com/backtrace-labs/bcd) (optional, for crash tracing)\n\nGenerally:\n```\nmake\nmake install\n```\n\nTo build without libbcd support:\n```\nNO_BCD=1 make\n```\n\n## Debugging\n\nFQ can be run in debug mode from the command line.\n\nTo run FQ in debug mode, kill any and all existing FQ processes, then enter the\nfollowing command:\n```\nfq -g fq FQ_DEBUG=\u003cflag values\u003e \u003cpath to fqd\u003e/fqd -D -c \u003cpath to fqd.sqlite\u003e/fqd.sqlite -p \u003cport number\u003e\n```\n\nFlag values determine debug output type and can have the following values:\n```\nFQ_DEBUG_MEM =     0x00000001,\nFQ_DEBUG_MSG =     0x00000002,\nFQ_DEBUG_ROUTE =   0x00000004,\nFQ_DEBUG_IO =      0x00000008,\nFQ_DEBUG_CONN =    0x00000010,\nFQ_DEBUG_CONFIG =  0x00000020,\nFQ_DEBUG        =  0x00000040,\nFQ_DEBUG_PEER =    0x00000080,\nFQ_DEBUG_HTTP =    0x00000100,\nFQ_DEBUG_PANIC =   0x40000000\n```\n\nTo debug more than one flag, simply OR the flag values. For example, to output\nconnection, configuration, and route information, set `FQ_DEBUG` equal to\n`0x00000034 (FQ_DEBUG_CONFIG|FQ_DEBUG_CONN|FQ_DEBUG_ROUTE)`.\n\nFor example, you can run FQ in debug mode with the variables shown below to\noutput configuration, connection, and route information to the console:\n```\nfq -g fq FQ_DEBUG=0x00000034  /opt/circonus/sbin/fqd -D -c /opt/circonus/var/lib/fq/fqd.sqlite -p 8765\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcirconus-labs%2Ffq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcirconus-labs%2Ffq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcirconus-labs%2Ffq/lists"}