{"id":17654953,"url":"https://github.com/aabouzaid/flume-formula","last_synced_at":"2025-03-30T09:26:40.459Z","repository":{"id":80895687,"uuid":"118507064","full_name":"aabouzaid/flume-formula","owner":"aabouzaid","description":"SaltStack formula for Apache Flume.","archived":false,"fork":false,"pushed_at":"2023-09-09T22:32:07.000Z","size":12,"stargazers_count":0,"open_issues_count":1,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-05T11:36:37.381Z","etag":null,"topics":["apache-flume","flume","saltstack","saltstack-formula"],"latest_commit_sha":null,"homepage":"http://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aabouzaid.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-22T19:52:05.000Z","updated_at":"2023-08-08T13:22:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"67742ed6-6e28-445f-a3d8-9ef877a20c0a","html_url":"https://github.com/aabouzaid/flume-formula","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aabouzaid%2Fflume-formula","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aabouzaid%2Fflume-formula/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aabouzaid%2Fflume-formula/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aabouzaid%2Fflume-formula/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aabouzaid","download_url":"https://codeload.github.com/aabouzaid/flume-formula/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246299094,"owners_count":20755106,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-flume","flume","saltstack","saltstack-formula"],"created_at":"2024-10-23T12:40:24.067Z","updated_at":"2025-03-30T09:26:40.439Z","avatar_url":"https://github.com/aabouzaid.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"============\nApache Flume\n============\nInstall and configure `Apache Flume \u003chttps://flume.apache.org/\u003e`_ a distributed, reliable,\nand available service for efficiently collecting, aggregating, and moving large amounts of log data.\n\n.. Note::\n\n    See the full `Salt Formulas installation and usage instructions\n    \u003chttp://docs.saltstack.com/en/latest/topics/development/conventions/formulas.html\u003e`_.\n\nAvailable states\n================\n\n.. contents::\n    :local:\n\n``flume``\n---------\n\nDownload, install, configure Apache Flume.\n\nConfiguration\n=============\n\nMain important sections are ``agents`` and ``plugins``.\n\nAgents\n------\nOne or more agents could be added under ``agents`` section.\nAnd each agent could have many sources, channels, sinks.\n\n.. code:: yaml\n\n    flume:\n\n      agents:\n\n        agent01:\n\n          sources:\n            httpSource:\n              type: http\n              port: 8080\n\n          channels:\n            httpChannel01:\n              type: memory\n              capacity: 1000000\n              transactionCapacity: 100000\n            nullChannel01:\n              type: memory\n              capacity: 1000\n              transactionCapacity: 1000\n\n          sinks:\n            httpFileSink01:\n              type: file_roll\n              channel: httpChannel01\n              sink:\n                directory: /var/log/flume\n                rollInterval: 300\n            nullSink01:\n              type: 'null'\n              channel: nullChannel01\n\n          sinkgroups:\n            sinkgroup01:\n              processor:\n                type: failover\n                backoff: 'false'\n                selector: failover\n\nPlugins\n-------\nMore Flume plugins (i.e. external libs) could be added under ``plugins`` section.\n\nIf the plugin doesn't follow `Flume convention\n\u003chttps://flume.apache.org/FlumeUserGuide.html#directory-layout-for-plugins\u003e`_,\nthen the non-standard dirs should be exposed in ``classpath`` section.\n\nIf the plugin does follow the convention, then no need to ``classpath`` section.\n\n.. code:: yaml\n\n    flume:\n\n      plugins:\n\n        hadoop:\n          source: https://archive.apache.org/dist/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz\n          hash: False\n          paths:\n            # No need to expose dirs here.\n            native:\n              src: lib/native\n              dest: native\n            hdfs:\n              src: share/hadoop/hdfs\n              dest: hdfs\n              # Expose classpath dirs,\n              # in case the plugin doesn't follow Flume plugins convention.\n              classpath:\n                - '*'\n                - 'lib/*'\n            common:\n              src: share/hadoop/common\n              dest: common\n              classpath:\n                - '*'\n                - 'lib/*'\n\n.. vim: fenc=utf-8 spell spl=en cc=100 tw=99 fo=want sts=4 sw=4 et\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faabouzaid%2Fflume-formula","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faabouzaid%2Fflume-formula","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faabouzaid%2Fflume-formula/lists"}