{"id":20564483,"url":"https://github.com/tarantool/xtm","last_synced_at":"2025-10-04T13:23:11.103Z","repository":{"id":46601330,"uuid":"331059788","full_name":"tarantool/xtm","owner":"tarantool","description":null,"archived":false,"fork":false,"pushed_at":"2021-10-08T11:22:14.000Z","size":54,"stargazers_count":2,"open_issues_count":1,"forks_count":2,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-01-16T19:44:31.828Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tarantool.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-19T17:37:47.000Z","updated_at":"2021-10-18T09:44:51.000Z","dependencies_parsed_at":"2022-08-24T21:11:55.828Z","dependency_job_id":null,"html_url":"https://github.com/tarantool/xtm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fxtm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fxtm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fxtm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fxtm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tarantool","download_url":"https://codeload.github.com/tarantool/xtm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242174173,"owners_count":20084149,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T04:27:09.422Z","updated_at":"2025-10-04T13:23:06.063Z","avatar_url":"https://github.com/tarantool.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# xtm - a library for cross-thread messaging with event loop integration\n\nThe library provides the following facilities:\n\n# xtm_queue\n\nOpaque struct, that represents unidirectional, single-writer-single-reader queue\nimplementation with event loop integration ability.\n\n## xtm_queue_new\n\nAllocation function, used to create instance of `struct xtm_queue` and return\nits pointer or `NULL` in case of error. Accepts the queue size as input, which\nmust be a power of two.\n\n## xtm_queue_delete\n\nDeallocation function, used to free queue and close its internal fds, in case\nwhen appropriate flags (`XTM_QUEUE_MUST_CLOSE_PRODUCER_READFD` and\n`XTM_QUEUE_MUST_CLOSE_CONSUMER_READFD`) are passed to this function.\nReturns 0 on success. Otherwise -1 with errno set appropriately (as in\n`close(2)`, since it implies close of fds)\n\n## xtm_queue_notify_consumer\n\nFunction for queue consumer notification.\nReturns 0 on success. Otherwise -1 with errno set appropriately (as in\n`write(2)`, since it implies write to internal fd)\n\n## xtm_queue_notify_producer\n\nFunction for queue producer notification.\nReturns 0 on success. Otherwise -1 with errno set appropriately (as in\n`write(2)`, since it implies write to internal fd)\n\n## xtm_queue_probe\n\nReturns 0 if queue has space. Otherwise -1 with errno set to `ENOBUFS`.\n\n## xtm_queue_count\n\nFunction returns current message count in queue.\n\n## xtm_queue_push_fun\n\nFunction puts message, which contains function and its argument to the queue.\nThis function does not notify the consumer thread, but only push to the queue.\nTo notify the consumer thread you must call `xtm_queue_notify_consumer`. The less\noften you notify, the greater the performance, but the greater the latency.\nFunction accepts flags, which define it's behaviour:\n`XTM_QUEUE_PRODUCER_NEEDS_NOTIFICATIONS` - if this flag is set, producer sets\nspecial flag in queue internals, if push fails. Consumer can check this flag,\nusing `xtm_queue_get_reset_was_full` function and notify producer.\nReturns 0 if queue has space. Otherwise -1 with errno set to `ENOBUFS`.\n\n## xtm_queue_consumer_fd\n\nReturns file descriptor, that should be watched by consumer thread to\nbecome readable. When it became readable, consumer should call one of\nthe consumer functions: `xtm_queue_pop_ptrs` or `xtm_queue_invoke_funs_all`.\n\n## xtm_queue_producer_fd\n\nReturn file descriptor, that should be watched by producer thread to become\nreadable in case when `xtm_queue_push_fun` or `xtm_queue_push_ptr` fails. When\nit became readable, producer may try push to the queue again. Very rarely,\nthere may be a situation (because of race between producer and consumer\nthreads), when the descriptor became readable, but queue is still full. In this\ncase producer thread needs to poll this descriptor again.\n\n## xtm_queue_invoke_funs_all\n\nFunction calls all functions contained in the queue at the time this function is\ncalled. We do not call those functions, that the producer thread can add at the\nmoment, when this function is already called, in order to prevent an infinite loop\nand hunger. Also, if the queue size is very large, then hunger is still possible,\nsince this function does not allow limiting the number of called functions.\nIf producer thread pushes functions with `XTM_QUEUE_PRODUCER_NEEDS_NOTIFICATIONS`\nflag, user should retrieve and reset \"producer failed to put an item\nin the queue and expects notification\" flag using `xtm_queue_get_reset_was_full`.\nIf this flag was true, user should notify producer thread (see simple example\nfurther).\nReturn count of invoked functions.\n\n## xtm_queue_push_ptr\n\nFunction puts message, which contains pointer to the queue. This function does not\nnotify the consumer thread, but only pushes to the queue. To notify the consumer thread\nyou must call `xtm_queue_notify_consumer`. The less often you notify, the greater the\nperformance, but the greater the latency.\nFunction accepts flags, which define it's behaviour:\n`XTM_QUEUE_PRODUCER_NEEDS_NOTIFICATIONS` - if this flag is set, producer sets\nspecial flag in queue internals, if push fails. Consumer can check this flag,\nusing `xtm_queue_get_reset_was_full` function and notify producer.\nReturns 0 if queue has space. Otherwise -1 with errno set to `ENOBUFS`.\n\n## xtm_queue_pop_ptrs\n\nFunction gets up to count elements from the queue and saves them in pointer array.\nIf producer thread pushes ptrs with `XTM_QUEUE_PRODUCER_NEEDS_NOTIFICATIONS`\nflag, user should retrieve and reset \"producer failed to put an item\nin the queue and expects notification\" flag using `xtm_queue_get_reset_was_full`.\nIf this flag was true, user should notify producer thread (see simple example further).\nReturn count of extracted messages.\n\n## xtm_queue_consume\n\nFunction reads from internal queue pipe, according to file descriptor passed\nto this function.\nReturn 0 on success. Otherwise -1 with errno set appropriately (as in `read(2)`,\nsince it implies read from internal fd).\n\n## xtm_queue_get_reset_was_full\n\nFunction retrieves and resets \"producer failed to put an item in the queue and\nexpects notification\" flag.\n\nExamples\n--------\n\nFull example you can see in test or perf folders, here is only simple\nexample of using library.\n\n** Allocate xtm queue **\n\n```c\nxtm_queue = xtm_queue_new(XTM_QUEUE_SIZE);\n```\n\n** Get internal fd on consumer side and pool it in the loop **\n\n```c\nint fd = xtm_queue_consumer_fd(xtm_queue);\nstruct pollfd pfds[1];\npfds[0].fd = fd;\npfds[0].events = POLLIN;\nint rc;\nwhile ((rc = poll(pfds, 1, -1)) \u003c 0 \u0026\u0026 errno == EINTR)\n\t;\nif (rc \u003c= 0 || (pfds[0].revents \u0026 POLLIN) == 0)\n\t//error handling\n```\n\n** Invoke functions in case when producer thread push functions **\n\n```c\nxtm_queue_invoke_funs_all(xtm_queue);\n```\n\n** Pop pointers to data in case when producer thread push pointers **\n\n```c\nunsigned count = xtm_queue_count(xtm_queue);\nunsigned cnt = 0;\nwhile (cnt \u003c count) {\n\tvoid *ptr_array[BATCH_COUNT_MAX];\n\tunsigned rc = xtm_queue_pop_ptrs(xtm_queue, ptr_array,\n\t\t\t\t\t BATCH_COUNT_MAX);\n\tfor (unsigned i = 0; i \u003c rc; i++)\n\t\t// do something with ptr_array[i]\n\tcnt += rc;\n}\n/* Try to notify producer again if queue was full, if it's fails - panic */\nif (xtm_queue_get_reset_was_full(xtm_queue) \u0026\u0026\n    xtm_queue_notify_producer(xtm_queue) != 0)\n\tpanic();\n```\n\n** Delete xtm queue, when it is no longer needed, close all internal fds **\n\n```c\nunsigned flags = 0;\nflags |= XTM_QUEUE_MUST_CLOSE_PRODUCER_READFD |\n         XTM_QUEUE_MUST_CLOSE_CONSUMER_READFD;\nxtm_queue_delete(xtm_queue, flags);\n```\n\n** Get internal fd on producer side, push fun to consumer thread **\n\n```c\nunsigned flags = XTM_QUEUE_PRODUCER_NEEDS_NOTIFICATIONS;\nint fd = xtm_queue_producer_fd(xtm_queue);\nwhile (xtm_queue_push_fun(xtm_queue, function, pointer, flags) != 0) {\n\tstruct pollfd pfds[1];\n\tpfds[0].fd = fd;\n\tpfds[0].events = POLLIN;\n\tint rc;\n\t// Wait for consumer notification, that queue is not full\n\twhile ((rc = poll(pfds, 1, -1)) \u003c 0 \u0026\u0026 errno == EINTR)\n\t\t;\n\tif (rc \u003c= 0 || (pfds[0].revents \u0026 POLLIN) == 0)\n\t\t//error handling\n\txtm_queue_consume(fd);\n}\n```\n\n** Get internal fd on producer side, push ptr to consumer thread **\n\n```c\nunsigned flags = XTM_QUEUE_PRODUCER_NEEDS_NOTIFICATIONS;\nint fd = xtm_queue_producer_fd(xtm_queue);\nwhile (xtm_queue_push_ptr(xtm_queue, pointer, flags) \u003c 0) {\n\tstruct pollfd pfds[1];\n\tpfds[0].fd = fd;\n\tpfds[0].events = POLLIN;\n\tint rc;\n\t// Wait for consumer notification, that queue is not full\n\twhile ((rc = poll(pfds, 1, -1)) \u003c 0 \u0026\u0026 errno == EINTR)\n\t\t;\n\tif (rc \u003c= 0 || (pfds[0].revents \u0026 POLLIN) == 0)\n\t\t//error handling\n\txtm_queue_consume(fd);\n}\n```\n\n** Wait in the loop until queue became not full **\n\n```c\nwhile (xtm_queue_probe(queue) != 0)\n\t;\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarantool%2Fxtm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftarantool%2Fxtm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarantool%2Fxtm/lists"}