{"id":28501948,"url":"https://github.com/fluent/fluent-plugin-multiprocess","last_synced_at":"2025-09-20T21:44:30.063Z","repository":{"id":10671448,"uuid":"12907117","full_name":"fluent/fluent-plugin-multiprocess","owner":"fluent","description":"Multiprocess agent plugin for Fluentd","archived":false,"fork":false,"pushed_at":"2022-09-27T00:00:17.000Z","size":16,"stargazers_count":44,"open_issues_count":7,"forks_count":12,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-06-08T16:08:27.874Z","etag":null,"topics":["fluentd","fluentd-plugin"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fluent.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-09-17T21:21:34.000Z","updated_at":"2025-01-04T00:28:30.000Z","dependencies_parsed_at":"2022-09-12T11:44:22.364Z","dependency_job_id":null,"html_url":"https://github.com/fluent/fluent-plugin-multiprocess","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/fluent/fluent-plugin-multiprocess","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-multiprocess","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-multiprocess/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-multiprocess/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-multiprocess/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fluent","download_url":"https://codeload.github.com/fluent/fluent-plugin-multiprocess/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-multiprocess/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263671746,"owners_count":23494027,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fluentd","fluentd-plugin"],"created_at":"2025-06-08T16:08:30.738Z","updated_at":"2025-09-20T21:44:25.005Z","avatar_url":"https://github.com/fluent.png","language":"Ruby","readme":"# Multiprocess agent plugin for Fluentd\n\n## multiprocess\n\n**multiprocess** agent plugin runs some child fluentd processes.\nYou can take advantage of multiple CPU cores using this plugin.\n\nPlease note that this plugin does not generate records although this is an input plugin.\nThis plugin just controls start \u0026 shutdown of child processes.\n\n\n### Configuration\n\n    \u003csource\u003e\n      @type multiprocess\n\n      # optional:\n      #graceful_kill_interval 2s\n      #graceful_kill_interval_increment 3s\n      #graceful_kill_timeout 60s\n\n      \u003cprocess\u003e\n        cmdline -c /etc/fluent/fluentd_child1.conf --log /var/log/fluent/fluentd_child1.log\n        pid_file /var/run/fluentd_child1.pid\n      \u003c/process\u003e\n      \u003cprocess\u003e\n        cmdline -c /etc/fluent/fluentd_child2.conf --log /var/log/fluent/fluentd_child2.log\n        sleep_before_start 5s\n        pid_file /var/run/fluentd_child2.pid\n      \u003c/process\u003e\n      \u003cprocess\u003e\n        cmdline -c /etc/fluent/fluentd_child3.conf --log /var/log/fluent/fluentd_child3.log\n        sleep_before_shutdown 5s\n        pid_file /var/run/fluentd_child3.pid\n      \u003c/process\u003e\n    \u003c/source\u003e\n\n- **process**: section sets command line arguments of a child process. This plugin creates one child process for each \\\u003cprocess\\\u003e section\n- **cmdline**: This parameter is required in a \\\u003cprocess\\\u003e section\n- **sleep\\_before\\_start**: Optional. Sets wait time before starting the process. Note that child processes **start from last to first** (`fluentd_child3` -\\\u003e `sleep 5` -\\\u003e { `fluentd_child2`, `fluentd_child1` } in this case)\n- **sleep\\_before\\_shutdown**: Optional. Sets wait time before shutting down the process. Note that child processes **shutdown from first to last** ({ `fluentd_child1`, `fluentd_child2` } -\\\u003e `sleep 5` -\u003e `fluentd_child3` in this case)\n- **pid_file**: Optional. Writes child process id to this file. This is useful for sending a signal to child processes, e.g. logrotate's  pid.\n\n### Logs for daemonized processes\n\nDaemonized fluentd closes its STDOUT. So child processes on daemonized fluentd \u0026 in_multiprocess doesn't put its logs without `--log` option. Specifing `--log` option always is best practice to help this situation.\n\n### NOTE\n\n* You should use unique parameter in each sub configuration to avoid conflict problem, e.g. buffer_path of `buf_file`, `s3_object_key_format` of `out_s3` and more.\n* This plugin is mainly for Fluentd v0.x, since Fluentd v1.x has [multi-process-workers feature](https://docs.fluentd.org/deployment/multi-process-workers).\nHowever, you can still use this plugin for Fluentd v1.x to run plugins that don't support the feature in multiple processes (with special care of the configuration to avoid race).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-plugin-multiprocess","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluent%2Ffluent-plugin-multiprocess","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-plugin-multiprocess/lists"}