{"id":13792510,"url":"https://github.com/tarantool/expirationd","last_synced_at":"2025-09-13T01:41:36.717Z","repository":{"id":17649130,"uuid":"20453639","full_name":"tarantool/expirationd","owner":"tarantool","description":"Expiration daemon module for Tarantool","archived":false,"fork":false,"pushed_at":"2025-09-12T09:14:14.000Z","size":300,"stargazers_count":48,"open_issues_count":12,"forks_count":14,"subscribers_count":27,"default_branch":"master","last_synced_at":"2025-09-12T09:24:58.029Z","etag":null,"topics":["expiration","lua","tarantool","tarantool-database"],"latest_commit_sha":null,"homepage":"https://tarantool.github.io/expirationd/","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tarantool.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2014-06-03T18:07:00.000Z","updated_at":"2025-02-10T01:47:57.000Z","dependencies_parsed_at":"2023-02-10T12:00:19.847Z","dependency_job_id":"5cd1a92e-4c51-4b14-b93f-e7b152679632","html_url":"https://github.com/tarantool/expirationd","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/tarantool/expirationd","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fexpirationd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fexpirationd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fexpirationd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fexpirationd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tarantool","download_url":"https://codeload.github.com/tarantool/expirationd/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tarantool%2Fexpirationd/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274907821,"owners_count":25371819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-12T02:00:09.324Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["expiration","lua","tarantool","tarantool-database"],"created_at":"2024-08-03T22:01:13.084Z","updated_at":"2025-09-13T01:41:36.669Z","avatar_url":"https://github.com/tarantool.png","language":"Lua","readme":"[![Run tests](https://github.com/tarantool/expirationd/actions/workflows/fast_testing.yaml/badge.svg)](https://github.com/tarantool/expirationd/actions/workflows/fast_testing.yaml)\n\u003ca href='https://coveralls.io/github/tarantool/expirationd?branch=master'\u003e\n\u003cimg src='https://coveralls.io/repos/github/tarantool/expirationd/badge.svg?branch=master' alt='Coverage Status' /\u003e\n\u003c/a\u003e\n\n# expirationd -  data expiration with custom quirks.\n\nThis package can turn Tarantool into a persistent memcache replacement,\nbut is powerful enough so that  your own expiration strategy can be defined.\n\nYou define two functions: one takes a tuple as an input and returns\ntrue in case it's expired and false otherwise. The other takes the\ntuple and performs the expiry itself: either deletes it (memcache), or\ndoes something smarter, like put a smaller representation of the data\nbeing deleted into some other space.\n\nThere are a number of similar modules:\n- [moonwalker](https://github.com/tarantool/moonwalker) triggered manually,\nuseful for batch transactions, a performance about 600/700k rec/sec\n- [expirationd](https://github.com/tarantool/expirationd/issues/53) always\nexpires tuples with using indices and using any condition, without guarantee\nfor time expiration.\n- [indexpirationd](https://github.com/moonlibs/indexpiration) always expires\ntuples with indices, has a nice precision (up to ms) for time to expire.\n\nTable below may help you to choose a proper module for your requirements:\n\n| Module        | Reaction time | Uses indices | Arbitrary condition | Expiration trigger                 |\n|---------------|---------------|--------------|---------------------|------------------------------------|\n| indexpiration | High (ms)     | Yes          | No                  | synchronous (fiber with condition) |\n| expirationd   | Medium (sec)  | Yes          | Yes                 | synchronous (fiber with condition) |\n| moonwalker    | NA            | No           | Yes                 | asynchronous (using crontab etc)   |\n\n### Prerequisites\n\n* Tarantool 1.10+ (`tarantool` package, see [documentation](https://www.tarantool.io/en/download/)).\n\n### Installation\n\nYou can:\n\n* Install the module using [tt](https://github.com/tarantool/tt):\n\n  ``` bash\n  tt rocks install expirationd\n  ```\n\n* Install the module using LuaRocks:\n\n  ``` bash\n  luarocks install --local --server=https://rocks.tarantool.org expirationd\n  ```\n\n### Documentation\n\nSee API documentation in https://tarantool.github.io/expirationd/\n\nNote about using expirationd with replication: by default expirationd processes\ntasks for all types of spaces only on the writable instance. It does not\nprocess tasks on read-only instance for [non-local persistent spaces](https://www.tarantool.io/en/doc/latest/reference/configuration/#confval-read_only).\nIt means that expirationd *will not* start task processing on a replica for\nregular spaces. One can force running task on replica with option `force` in\n`start()` module function. The option force let a user control where to start\ntask processing and where don't.\n\n### Examples\n\nSimple version:\n\n```lua\nbox.cfg{}\nspace = box.space.old\njob_name = \"clean_all\"\nexpirationd = require(\"expirationd\")\n\nfunction is_expired(args, tuple)\n  return true\nend\n\nfunction delete_tuple(space, args, tuple)\n  box.space[space]:delete{tuple[1]}\nend\n\nexpirationd.start(job_name, space.id, is_expired, {\n    process_expired_tuple = delete_tuple,\n    args = nil,\n    tuples_per_iteration = 50,\n    full_scan_time = 3600\n})\n```\n\nСustomized version:\n\n```lua\nexpirationd.start(job_name, space.id, is_expired, {\n    -- name or id of the index in the specified space to iterate over\n    index = \"exp\",\n    -- one transaction per batch\n    -- default is false\n    atomic_iteration = true,\n    -- delete data that was added a year ago\n    -- default is nil\n    start_key = function( task )\n        return clock.time() - (365*24*60*60)\n    end,\n    -- delete it from the oldest to the newest\n    -- default is ALL\n    iterator_type = \"GE\",\n    -- stop full_scan if delete a lot\n    -- returns true by default\n    process_while = function( task )\n        if task.args.max_expired_tuples \u003e= task.expired_tuples_count then\n            task.expired_tuples_count = 0\n            return false\n        end\n        return true\n    end,\n    -- this function must return an iterator over the tuples\n    iterate_with = function( task )\n        return task.index:pairs({ task.start_key() }, { iterator = task.iterator_type })\n            :take_while( function( tuple )\n                return task:process_while()\n            end )\n    end,\n    args = {\n        max_expired_tuples = 1000\n    }\n})\n```\n\n## Testing\n\n```\n$ make deps-full\n$ make test\n```\n\nRegression tests running in continuous integration that uses luatest are\nexecuted in shuffle mode. It means that every time order of tests is\npseudorandom with predefined seed. If tests in CI are failed it is better to\nreproduce these failures with the same seed:\n\n```sh\n$ make SEED=1334 test\nluatest -v --coverage --shuffle all:1334\n...\n```\n\n## Cartridge role\n\n`cartridge.roles.expirationd` is a Tarantool Cartridge role for the expirationd\npackage with features:\n\n* It registers expirationd as a Tarantool Cartridge service for easy access to\n  all [API calls](https://tarantool.github.io/expirationd/#Module_functions):\n  ```Lua\n  local task = cartridge.service_get('expirationd').start(\"task_name\", id, is_expired)\n  task:kill()\n  ```\n* You could configure the expirationd role with `cfg` entry.\n  [expirationd.cfg()](https://tarantool.github.io/expirationd/#cfg) has the\n  same parameters with the same meaning.\n\n  Be careful, values from the clusterwide configuration are applied by default\n  to all nodes on each\n  [apply_config()](https://www.tarantool.io/en/doc/latest/book/cartridge/cartridge_dev/).\n  Changing the configuration manually with\n  [expirationd.cfg()](https://tarantool.github.io/expirationd/#cfg)\n  only affects the current node and does not update values in the clusterwide\n  configuration. The manual change will be overwritten by a next\n  `apply_config` call.\n* You can use persistent functions (i.e. created by `box.schema.func.create`).\n  When configuring, role tries firstly get function from global namespace\n  (`_G`) and if function was not found then role tries search in `box.func` for\n  function with the same name.\n\n  Be careful! At the moment of validating and applying config of expirationd\n  role all persistent functions must be created before, so to configure\n  cartridge application correctly you must do it in two steps: at the first\n  step you have to confgure migrations with creating persistent functions and\n  run them, at the second one put expirationd config.\n* The role stops all expirationd tasks on an instance on the role termination.\n* The role can automatically start or kill old tasks from the role\n  configuration:\n\n  ```yaml\n  expirationd:\n    cfg:\n      metrics: true\n    task_name1:\n      space: 579\n      is_expired: is_expired_func_name_in__G\n      is_master_only: true\n      options:\n        args:\n          - any\n        atomic_iteration: false\n        force: false\n        force_allow_functional_index: true\n        full_scan_delay: 1\n        full_scan_time: 1\n        index: 0\n        iterate_with: iterate_with_func_name_in__G\n        iteration_delay: 1\n        iterator_type: ALL\n        on_full_scan_complete: on_full_scan_complete_func_name_in__G\n        on_full_scan_error: on_full_scan_error_func_name_in__G\n        on_full_scan_start: on_full_scan_start_func_name_in__G\n        on_full_scan_success: on_full_scan_success_func_name_in__G\n        process_expired_tuple: process_expired_tuple_func_name_in__G\n        process_while: process_while_func_name_in__G\n        start_key:\n        - 1\n        tuples_per_iteration: 100\n        vinyl_assumed_space_len: 100\n        vinyl_assumed_space_len_factor: 1\n    task_name2:\n      ...\n  ```\n\n  [expirationd.start()](https://tarantool.github.io/expirationd/#start) has\n  the same parameters with the same meaning except for the additional optional\n  param `is_master_only`. If `true`, the task should run only on a master\n  instance. By default, the value is `false`.\n\n  You need to be careful with parameters-functions. The string is a key in\n  the global variable `_G`, the value must be a function. You need to define\n  the key before initializing the role:\n\n  ```Lua\n  rawset(_G, \"is_expired_func_name_in__G\", function(args, tuple)\n      -- code of the function\n  end)\n  ```\n\n## Tarantool 3.0 role\n\n`roles.expirationd` is a Tarantool 3.0 role for the expirationd\npackage with the following features:\n\n* You can configure the expirationd role with `cfg` entry (check example).\n  Cluster configuration allows to set the same parameters as\n  in [expirationd.cfg()](https://tarantool.github.io/expirationd/#cfg)\n* You can use persistent functions (i.e. created by `box.schema.func.create`)\n  for expirationd `cfg` entries.\n  When configuring, role tries first to get a function from global namespace (`_G`)\n  and if the function was not found then role tries to search in `box.func`\n  for a function with the same name.\n  If some functions from config are missing,\n  expirationd will wait for their creation and start tasks when all of them are found.\n  You can check logs to see what functions are missing.\n* The role stops all expirationd tasks on an instance on the role termination.\n* The role can automatically start or kill old tasks from the role\n  configuration.\n\n  ```yaml\n  roles: [roles.expirationd]\n  roles_cfg:\n    roles.expirationd:\n      cfg:\n        metrics: true\n      task_name1:\n        space: users\n        is_expired: is_expired_func_name\n        is_master_only: true\n        options:\n          args:\n            - any\n          atomic_iteration: false\n          force: false\n          force_allow_functional_index: true\n          full_scan_delay: 1\n          full_scan_time: 1\n          index: 0\n          iterate_with: iterate_with_func_name_in__G\n          iteration_delay: 1\n          iterator_type: ALL\n          on_full_scan_complete: on_full_scan_complete_func_name_in__G\n          on_full_scan_error: on_full_scan_error_func_name_in__G\n          on_full_scan_start: on_full_scan_start_func_name_in__G\n          on_full_scan_success: on_full_scan_success_func_name_in__G\n          process_expired_tuple: process_expired_tuple_func_name_in__G\n          process_while: process_while_func_name_in__G\n          start_key:\n            - 1\n          tuples_per_iteration: 100\n          vinyl_assumed_space_len: 100\n          vinyl_assumed_space_len_factor: 1\n  ```\n\n  [expirationd.start()](https://tarantool.github.io/expirationd/#start) has\n  the same parameters with the same meaning except for the additional optional\n  param `is_master_only`. If `true`, the task should run only on a master\n  instance. By default, the value is `false`.\n\n  You need to be careful with function parameters. Task will not start until it\n  finds all functions from config. You can define them in user code:\n\n  ```Lua\n  box.schema.func.create('is_expired_func_name', {\n      body = \"function(...) return true end\",\n      if_not_exists = true\n  })\n\n  -- Or you could define a global variable.\n  rawset(_G, \"process_while_func_name_in__G\", function(...)\n      return true\n  end)\n  ```\n\n","funding_links":[],"categories":["Packages"],"sub_categories":["Database"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarantool%2Fexpirationd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftarantool%2Fexpirationd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftarantool%2Fexpirationd/lists"}