{"id":18707507,"url":"https://github.com/sumologic/sumologic-collectd-plugin","last_synced_at":"2025-04-12T10:24:11.188Z","repository":{"id":23739137,"uuid":"99730029","full_name":"SumoLogic/sumologic-collectd-plugin","owner":"SumoLogic","description":"Sumo Logic collectd Plugin: A collectd output plugin to send Carbon 2.0-formatted metrics to Sumo Logic.","archived":false,"fork":false,"pushed_at":"2024-03-22T06:19:05.000Z","size":217,"stargazers_count":8,"open_issues_count":4,"forks_count":5,"subscribers_count":33,"default_branch":"main","last_synced_at":"2024-04-14T06:09:14.456Z","etag":null,"topics":["collectd-plugin","sumologic"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SumoLogic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-08T19:44:25.000Z","updated_at":"2024-06-10T10:59:01.214Z","dependencies_parsed_at":"2024-06-10T10:58:55.352Z","dependency_job_id":"27202a74-af58-47ce-a274-9bebead5ddde","html_url":"https://github.com/SumoLogic/sumologic-collectd-plugin","commit_stats":{"total_commits":257,"total_committers":8,"mean_commits":32.125,"dds":0.2957198443579766,"last_synced_commit":"f8b8314055547f26bd04ebd790787563efaab4b9"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SumoLogic%2Fsumologic-collectd-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SumoLogic%2Fsumologic-collectd-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SumoLogic%2Fsumologic-collectd-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SumoLogic%2Fsumologic-collectd-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SumoLogic","download_url":"https://codeload.github.com/SumoLogic/sumologic-collectd-plugin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248551548,"owners_count":21123114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["collectd-plugin","sumologic"],"created_at":"2024-11-07T12:18:31.145Z","updated_at":"2025-04-12T10:24:11.138Z","avatar_url":"https://github.com/SumoLogic.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sumo Logic collectd Plugin\n\nA [collectd](https://collectd.org/) output plugin to send Carbon 2.0-formatted metrics to Sumo Logic.\n\n## Get Started\n\n### 1. Python version\n\nSumo Logic collectd plugin is built on top of [collectd-python plugin](https://collectd.org/documentation/manpages/collectd-python.5.shtml).\nThe minimum required version for running the plugin is Python version 2.6.\nYou can download and install the desired Python version from [Python download page](https://www.python.org/downloads/).\n\n### 2. Install collectd on your machine\n\nIf collectd is already installed, you can skip this step.\nOtherwise, follow the instructions in the [collectd download](https://collectd.org/download.shtml) site for download and installation.\nFor additional details, see [First steps](https://collectd.org/wiki/index.php/First_steps) section in the collectd Wiki.\n\n#### MacOS\n\n```sh\nbrew install collectd\n```\n\n#### Debian / Ubuntu\n\n```sh\nsudo apt-get install collectd\n```\n\n### 3. Download and install Sumo Logic collectd plugin\n\n#### Option 1: Install as a library\n\n```sh\nsudo pip install sumologic_collectd_metrics\n```\n\nAll required dependencies will be automatically installed with library installation.\n\n#### Option 2: Install with source code\n\nThe Sumo Logic collectd plugin source code can be saved in a directory anywhere on your system.\nDownload it from [Python Package Index](https://pypi.python.org/pypi/sumologic_collectd_metrics),\nor:\n\n```sh\ngit clone https://github.com/SumoLogic/sumologic-collectd-plugin.git\n```\n\nSumo Logic collectd plugin uses [requests](http://docs.python-requests.org/en/master/)\nand [retry](https://pypi.python.org/pypi/retrying) libraries for sumbitting HTTPS requests.\nIf they are not installed. Install them using pip:\n\n```sh\nsudo pip install requests\nsudo pip install retry\n```\n\n### 4. Create an HTTP Metrics Source in Sumo Logic\n\nCreate a [Sumo Logic account](https://www.sumologic.com/) if you don't currently have one.\n\nFollow these instructions for [setting up an HTTP Source](https://help.sumologic.com/Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source/zGenerate-a-new-URL-for-an-HTTP-Source)\nin Sumo Logic.\nBe sure to obtain the URL endpoint after creating an HTTP Source.\n\n### 5. Configure Sumo Logic collectd plugin\n\nSumo Logic collectd plugin supports the following parameters.\nTo configure the plugin, modify collectd's configuration file named `collectd.conf` (e.g. `/etc/collectd/collectd.conf`).\n\n#### Required parameters\n\nThe parameters below are required and must be specified in the module\nconfig.\n\n| Name                                                                                                                                   | Description                                                                                                                                                                                                                              | Type   | Required |\n| -------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | -------- |\n| [URL](https://help.sumologic.com/Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source/zGenerate-a-new-URL-for-an-HTTP-Source) | The URL to send metrics to. This should be given when [creating the HTTP Source](https://help.sumologic.com/Send-Data/Sources/02Sources-for-Hosted-Collectors/HTTP-Source/zGenerate-a-new-URL-for-an-HTTP-Source) on Sumo Logic web app. | String | True     |\n\n#### Basic parameters\n\nThe parameters below are not strictly required. It is recommended to set\nthese parameters as they prove to be extremely useful to categorize your\nmetrics and search by them.\n\n| Name                     | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | Type                                                       | Required |\n| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------- | -------- |\n| SourceName               | Name of the metrics source. `_sourceName` can be used to search metrics from this source. It will override the default configured in the the Sumo Logic Source configuration.                                                                                                                                                                                                                                                                                                  | String                                                     | False    |\n| SourceHost               | Name of metrics host. `_sourceHost` can be used to search metrics from this host. It will override the default configured in the the Sumo Logic Source configuration.                                                                                                                                                                                                                                                                                                          | String                                                     | False    |\n| SourceCategory           | Category of the collected metrics. `_sourceCategory` can be used to search metrics from this category. It will override the default configured in the the Sumo Logic Source configuration.                                                                                                                                                                                                                                                                                     | String                                                     | False    |\n| Dimensions               | Key value pairs that contribute to identifying a metric. Collectd data have intrinsic dimensions with keys as `host`, `plugin`, `plugin_instance`, `type`, `type_instance`, `ds_name`, `ds_type`. The Additional dimensions specified here can help separating metrics collected from this collectd instance with metircs collected from other collectd instances. Dimensions cannot contain [Reserved symbols](#reserved-symbols) and [Reserved keywords](#reserved-keywords) | Srings in the format of `\"key1\"=\"val1\", \"key2\"=\"val2\"` ... | False    |\n| Metadata                 | Key value pairs that do not contribute to identifying a metric. Metadata are primarily used to assist in searching metrics. Collectd data may have internal metadata. The additional metadata specified here can be used to enrich the existing metadata set. Metadata cannot contain [Reserved symbols](#reserved-symbols) and [Reserved keywords](#reserved-keywords)                                                                                                        | Srings in the format of `\"key1\"=\"val1\", \"key2\"=\"val2\"` ... | False    |\n| AddMetricDimension       | If set to `true` adds new dimension named `metric` to every data point which consists of `type` and `type_instance` concatenated using `MetricDimensionSeparator`. By default it is set to `True`.                                                                                                                                                                                                                                                                             | Boolean                                                    | False    |\n| MetricDimensionSeparator | String used to concatenate `type` and `type_instance` while creating `metric` dimension. See `AddMetricDimension` option. By default it is `.`.                                                                                                                                                                                                                                                                                                                                | String                                                     | False    |\n\n#### Additional parameters\n\nFor additional configuration parameters, see [Advanced parameters](#advanced-parameters) below.\n\n#### Example configuration\n\nAn example configuration for the plugin is shown below (code to be added to `collectd.conf` under `collectd_root/etc`):\n\n```aconf\nLoadPlugin python\n\u003cPlugin python\u003e\n    # Uncomment and update the following line if sumologic collectd plugin installed with source code\n    # ModulePath \"/path/to/sumologic-collectd-plugin\"\n    LogTraces true\n    Interactive false\n    Import \"sumologic_collectd_metrics\"\n\n    \u003cModule \"sumologic_collectd_metrics\"\u003e\n        URL \"https://\u003cdeployment\u003e.sumologic.com/receiver/v1/http/\u003csource_token\u003e\"  # URL must be specified\n        # Uncomment and update the following lines to override the default metadata configured in the the Sumo Logic Source configuration\n        # SourceName \"my_source\"\n        # SourceHost \"my_host\"\n        # SourceCategory \"my_category\"\n        # Uncomment and update the following lines to add additional key=value pairs\n        # Dimensions \"my_dim_key1\"=\"my_dim_val1\", \"my_dim_key2\"=\"my_dim_val2\"\n        # Metadata \"my_meta_key1\"=\"my_meta_val1\", \"my_meta_key2\"=\"my_meta_val2\"\n    \u003c/Module\u003e\n\u003c/Plugin\u003e\n```\n\n#### Other recommended modules\n\nIt is recommeded to setup the following two plugins in `collectd.conf`.\nThe functionalities of the two plugins are explained in collectd Wiki\n[Plugin:LogFile](https://collectd.org/wiki/index.php/Plugin:LogFile)\nand [Plugin:CSV](https://collectd.org/wiki/index.php/Plugin:CSV):\n\n```aconf\nLoadPlugin logfile\n\u003cPlugin logfile\u003e\n    LogLevel \"info\"\n    File \"/var/log/collectd.log\"\n    Timestamp true\n    PrintSeverity true\n\u003c/Plugin\u003e\n\nLoadPlugin csv\n\u003cPlugin csv\u003e\n    DataDir \"/usr/local/var/lib/collectd/csv\"\n\u003c/Plugin\u003e\n```\n\nThe following pulgins, if enabled in `collectd.conf`, enable collecting\n[cpu](https://collectd.org/wiki/index.php/Plugin:CPU),\n[memory](https://collectd.org/wiki/index.php/Plugin:Memory),\n[disk](https://collectd.org/wiki/index.php/Plugin:Disk),\n[network](https://collectd.org/wiki/index.php/Plugin:Interface)\nmetrics from the system:\n\n```aconf\nLoadPlugin cpu\nLoadPlugin memory\nLoadPlugin disk\nLoadPlugin interface\n```\n\nA list of all collectd plugins is awailable in collectd Wiki [Table of\nPlugins](https://collectd.org/wiki/index.php/Table_of_Plugins)\n\n#### Reserved symbols\n\nEqual sign and space are reserved symbols.\n\n```console\n\"=\" \" \"\n```\n\n#### Reserved keywords\n\nFollowing terms are reserved for Sumo Logic internal use only:\n\n```console\n\"_sourcehost\", \"_sourcename\", \"_sourcecategory\", \"_collectorid\", \"_collector\", \"_source\", \"_sourceid\", \"_contenttype\", \"_rawname\"\n```\n\n### 6. Start sending metrics\n\nStart sending metrics by running collectd, e.g. (command will differ\ndepending on collectd installation):\n\n```sh\nsudo service collectd start\n```\n\n#### View logs\n\nIf `logfile` plugin is installed, then you can view logs by tailling collectd.log\nfile, e.g. (command can be different depending on collectd installation):\n\n```sh\ntail -f /var/log/collectd.log\n```\n\n#### Data model\n\nThe Sumo Logic collectd plugin will send metrics using the [Carbon 2.0](http://metrics20.org/implementations/) format, defined as:\n\n```console\ndimensions  metadata value timestamp\n```\n\n`dimensions` and `metadata` are key/value pairs of strings separated by two spaces.\n`dimensions` uniquely identifying a metric, while `metadata` do not contribute to identifying a metric.\nInstead, they are used to categorize metrics for searching.\n`value` is a double number.\n`timestamp` is a 10-digit UNIX epoch timestamp.\n\nExample data before compression:\n\n```console\nhost=my_mac plugin=cpu plugin_instance=1 type=cpu type_instance=user ds_name=value ds_type=DERIVE  meta_key1=meta_val1 5991.000000 1502148249\nhost=my_mac plugin=cpu plugin_instance=0 type=cpu type_instance=user ds_name=value ds_type=DERIVE  meta_key1=meta_val1 98722.000000 1502148249\n```\n\n#### Naming Schema\n\nCollectd uses a very powerful naming schema to identify each statistics value.\nIt has been proven very generic and flexible, but may be confusing at first, especially to new users.\nYou can read more about it by following this wiki [collectd naming schema](https://collectd.org/wiki/index.php/Naming_schema).\n\n#### Compression\n\nMetrics are batched and compressed before they are sent.\nThe default compression algorithm is `\"deflate\"`.\nThe algorithm is explained in more detail in [An Explanation of the Deflate Algorithm](https://zlib.net/feldspar.html).\nAlternatively, you can specify `\"gzip\"` for gzip compression and `\"none\"` for no compression.\n\n#### Error handling\n\nSumo Logic collectd plugin retries on exceptions by default.\nWhen all retries fail, the request is either scheduled for a future attempt or dropped based on the buffer status.\nBy default, 1000 requests are buffered.\nIf the buffer becomes full, then requests failed after all retries will be dropped.\nOtherwise, it is put back to the processing queue for the next run.\n\n### 7. View metrics\n\nTo view the metrics sent by the collectd plugin, log into Sumo Logic and open a Metrics tab.\nQuery for metrics using either dimensions or metadata, e.g.:\n\n```console\n_sourceName=my_source _sourceHost=my_host _sourceCategory=my_category plugin=cpu\n```\n\nYou should be able to see metrics displayed in the main graph.\n\n## Advanced Topics\n\n### Advanced parameters\n\nYou can configure the Sumo Logic collectd plugin by overriding default\nvalues for plugin parameters.\n\n| Name                  | Description                                                                                                                                                                                                                                                                                                                                                                          | Type                 | Default | Unit   |\n| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------- | ------- | ------ |\n| MaxBatchSize          | Sumo Logic collectd plugin batches metrics before sending them over https. MaxBatchSize defines the upper limit of metrics per batch.                                                                                                                                                                                                                                                | Positive Integer     | 5000    | NA     |\n| MaxBatchInterval      | Sumo Logic collectd plugin batches metrics before sending them through https. MaxBatchInterval defines the upper limit of duration to construct a batch.                                                                                                                                                                                                                             | Positive Integer     | 1       | Second |\n| HttpPostInterval      | Sumo Logic collectd plugin schedules https post requests at fixed intervals. HttpPostInterval defines the frequency for the scheduler to run. If no metrics batch is available at the time, the sceduler immediately returns. If multiple metrics batches are available, then the oldest batch is picked to be sent.                                                                 | Positive Float       | 0.1     |        |\n| MaxRequestsToBuffer   | Sumo Logic collectd plugin buffers failed and delayed metrics batch requests. MaxRequestsToBuffer specifies the maximum number of these requests to buffer. After the buffer becomes full, the request with oldest metrics batch will be dropped to make space for new metrics batch.|Positive Integer                                                                               | Positive Integer     | 1000    | NA     |\n| RetryInitialDelay     | Sumo Logic collectd plugin retries on recoverable exceptions. RetryInitialDelay specifies the initial delay before a retry is scheduled. More information can be found in the retry library                                                                                                                                                                                          | Non-negative Integer | 0       | Second |\n| RetryMaxAttempts      | Sumo Logic collectd plugin retries on recoverable exceptions. RetryMaxAttempts specifies the upper limit of retries before the current retry logic fails. The metric batch is then either put back for the next run (when metrics buffer specified by MaxRequestsToBuffer is not full), or dropped (when metrics buffer is full). More information can be found in the retry library | Positive Integer     | 10      | NA     |\n| RetryMaxDelay         | Sumo Logic collectd plugin retries on recoverable exceptions. RetryMaxDelay specifies the upper limit of delay before the current retry logic fails. Then the metric batch either is put back for the next run (when metrics buffer specified by MaxRequestsToBuffer is not full), or dropped (when metrics buffer is full). More information can be found in the retry library      | Positive Integer     | 100     | Second |\n| RetryJitterMin        | Sumo Logic collectd plugin retries on recoverable exceptions. RetryJitterMin specifies the minimum extra seconds added to delay between attempts. More information can be found in the retry library                                                                                                                                                                                 | Non-negative Integer | 0       | Second |\n| RetryJitterMax        | Sumo Logic collectd plugin retries on recoverable exceptions. RetryJitterMax specifies the maximum extra seconds added to delay between attempts. More information can be found in the retry library                                                                                                                                                                                 | Non-negative Integer | 10      | Second |\n| EnableInternalMetrics | Enable the plugin's internal metrics. See [Plugin Internal Metrics](#plugin-internal-metrics)                                                                                                                                                                                                                                                                                        | Boolean              | false   | NA     |\n| SignalFxStatsDTags    | Enable extracting metadata from metrics using SignalFx' extension to the StatsD protocol. See: https://docs.signalfx.com/en/latest/integrations/agent/monitors/collectd-statsd.html#adding-dimensions-to-statsd-metrics                                                                                                                                                              | Boolean              | false   | NA     |\n\n### Plugin Architecture\n\n```console\nCollectd        MetricsConverter                    MetricsBatcher         MetricsBuffer                               MetricsSender\n--------        --------------------------          --------------         ------------------------                    -----------------\n                                                                                                       batch to send   \nRaw Data   -\u003e   Metric in Carbon 2.0 format    -\u003e   Metrics Batch     -\u003e   Buffered metrics batches         -\u003e         Request scheduler\n                                                                                                            \u003c-\n                                                                                                       failed batch\n```\n\n### Plugin Internal Metrics\n\nIf the `EnableInternalMetrics` configuration option is set to true, the plugin emits metrics describing its own internal state.\n\nThe following metrics are available:\n\n| Name             | Type  | Description                                                                                                                          |\n|------------------|-------|--------------------------------------------------------------------------------------------------------------------------------------|\n| received_metrics | Gauge | The number of metrics values received from collectd. It will never decrease.                                                         |\n| sent_metrics     | Gauge | The number of metrics values successfully sent to Sumo. It will never decrease.                                                      |\n| sent_batches     | Gauge | The number of metrics batches successfully sent to Sumo. It will never decrease.                                                     |\n| dropped_metrics  | Gauge | The number of metrics values dropped during processing due to various faults, typically buffers being full. It will never decrease.  |\n| dropped_batches  | Gauge | The number of metrics batches dropped during processing due to various faults, typically buffers being full. It will never decrease. |\n| batch_queue_size | Gauge | The number of batches in the sending queue.                                                                                          |\n\n## TLS 1.2 Requirement\n\nSumo Logic only accepts connections from clients using TLS version 1.2 or greater.\nTo utilize the content of this repo, ensure that it's running in an execution environment that is configured to use TLS 1.2 or greater.\n\n## License\n\nThe Sumo Logic collectd output plugin is published under the Apache Software License, Version 2.0.\nPlease visit http://www.apache.org/licenses/LICENSE-2.0.txt for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumologic%2Fsumologic-collectd-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsumologic%2Fsumologic-collectd-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsumologic%2Fsumologic-collectd-plugin/lists"}