{"id":26057642,"url":"https://github.com/wavefronthq/wavefront-fdb-tailer","last_synced_at":"2025-04-11T05:06:39.807Z","repository":{"id":40463412,"uuid":"160748677","full_name":"wavefrontHQ/wavefront-fdb-tailer","owner":"wavefrontHQ","description":"Wavefront Metrics Adaptor for FoundationDB Logs","archived":false,"fork":false,"pushed_at":"2024-02-16T13:56:45.000Z","size":85,"stargazers_count":16,"open_issues_count":1,"forks_count":5,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-11T05:06:34.389Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wavefrontHQ.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-07T00:18:48.000Z","updated_at":"2025-04-09T10:13:38.000Z","dependencies_parsed_at":"2024-10-26T13:09:20.540Z","dependency_job_id":null,"html_url":"https://github.com/wavefrontHQ/wavefront-fdb-tailer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wavefrontHQ%2Fwavefront-fdb-tailer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wavefrontHQ%2Fwavefront-fdb-tailer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wavefrontHQ%2Fwavefront-fdb-tailer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wavefrontHQ%2Fwavefront-fdb-tailer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wavefrontHQ","download_url":"https://codeload.github.com/wavefrontHQ/wavefront-fdb-tailer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248345265,"owners_count":21088244,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-08T11:58:12.472Z","updated_at":"2025-04-11T05:06:39.763Z","avatar_url":"https://github.com/wavefrontHQ.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FoundationDB Metrics\n\nThe fdb-metrics Java application parses FoundationDB logs into metrics and sends them to Wavefront using a [Wavefront Proxy](https://docs.wavefront.com/proxies.html) or [Wavefront direct ingestion](https://docs.wavefront.com/direct_ingestion.html).  It can also be configured to send metrics to a Graphite server by using a [Graphite Reporter](https://metrics.dropwizard.io/3.1.0/manual/graphite/).\n\n#### What is Wavefront?\n[Wavefront](https://wavefront.com) is a SaaS-based metrics and observability platform.  More info can be found on the website linked, where you can also schedule a [30-day free trial](https://www.wavefront.com/sign-up/) or a [one-on-one demo](https://www.wavefront.com/schedule-a-meeting/)\n\n## Installation\nThis application is built from a maven repository. Simply run ```mvn clean install``` from the cloned repo and run the resulting jar.\n\n\n## Usage\n\n### Configuration\n\nThe fdb-metrics application can be configured via a YAML configuration file specified at the command line, or it can be configured entirely via command line arguments.\n\n### Command Line Configuration\nThe command line argument options are as follows:\n\n```\n--dir, -d\n     Absolute path to search for FDB logs.\n     \n--file, -f\n     Path to the yaml config file that specifies the parameters, if\n     not passed in via command line.\n     \n--graphitePort\n     The port for the Graphite Server.  Used only if --type is set to\n     GRAPHITE.\n\n--graphiteServer\n     The name of the machine running a Graphite server.  Used only if --type\n     is set to GRAPHITE.\n     \n--help, -h\n     Prints available options.\n     \n--matching, -m\n     A regex expression to match against potential log files in the directory.\n     Default: .*\n     \n--prefix\n     A prefix to attach to all metrics collected.  The default is \"fdb.trace.\" if not specified.\n     \n--proxyHost\n     The name of the machine running a Wavefront proxy.  Used only if --type\n     is set to PROXY.\n     Default: localhost\n     \n--proxyPort\n     The port the Wavefront proxy is listening on.  Used only if --type is set\n     to PROXY.\n     Default: 2878\n     \n--server\n     The name of the machine for Wavefront direct ingestion.  Used only if\n     --type is set to DIRECT.\n     \n--token\n     The API token for Wavefront direct ingestion. Used only if --type is set to\n     DIRECT.\n\n--endPoints\n    A list of Wavefront endpoints to send metrics. Used only if --type is set to DIRECT.\n\n--serviceName\n    Optional variable to control the name of the service reported. Defaults to fdbtailer.\n     \n--type\n     The type of reporter that should be used to report the metrics gathered.\n     Possible Values: [DIRECT, PROXY, GRAPHITE]\n\n--disabledMetrics\n    Option to disable certain metrics collected by FDBTailer.\n    Possible Values: [role, machineMetrics, processMetrics, storageMetrics, masterCommit, rkUpdate, totalDataInFlight, movingData, machineLoadDetail, programStart, memSample, memSampleSummary]\n\n```\n\n### Configuration via YAML\nYou can configure the application by specifying a YAML configuration file at the command line, like this:\n```\n    --file \u003cfile_path\u003e\n```\n\nAll of the potentional YAML configuration options correspond to command line options, and are listed here:\n\n```\ndirectory:\ngraphitePort:\ngraphiteServer:\nmatching:\nprefix:\nproxyHost:\nproxyPort:\nreporterType:\nserver:\ntoken:\nendPoints:\nserviceName\n```\n\nAn example YAML configuration file is included as example_config.yaml in this repo, and is reproduced here:\n\n```\ndirectory: \"/usr/local/foundationdb/logs\"\nmatching: \".*\\\\.xml$\"\nreporterType: PROXY\nproxyHost: 127.0.0.1\nproxyPort: 2878\ndisabledMetrics:\n  - \"machineLoadDetail\"\n  - \"memSampleSummary\"\n```\n\nThis is a simple configuration that sets up the fdb-metrics application to send metrics to a Wavefront proxy running on the local machine.  It will examine all files ending with ```.xml``` in the directory ```/usr/local/foundationdb/logs```.\n\n## Using Different Reporters\nCurrently the application supports three different metric reporters: the Wavefront Proxy reporter, the Wavefront direct ingestion reporter, and the Graphite reporter.  Three things need to be specified regardless of which reporter is being used:\n  * The ```directory```, the absolute path to search for logs\n  * The ```matching``` regex pattern to match files in that directory against\n  * The ```reporterType``` to specify which reporter should be used.\n\nIn addition, you have to specify two reporter-specific arguments for each of the different reporters, which are described in the next sections.\n### Using the Wavefront Proxy Reporter\n\nThe Wavefront Proxy reporter requires a [Wavefront Proxy](https://docs.wavefront.com/proxies.html) to be running and accessible.  You will need to provide the proxy address as ```proxyHost``` and the port the proxy is listening on as ```proxyPort```. If not provided, the default values will send to 127.0.0.1 at port 2878. \nHere is an example of the command line options that configure the application to send metrics to a Wavefront Proxy running on the local machine.\n\n```\n    --proxyHost 127.0.0.1\n    --dir \"/usr/local/foundationdb/logs\"\n    --matching \".*\\\\.xml$\"\n    --type PROXY\n```\n\n### Using the Direct Ingestion Reporter\n\nIn order to use the direct ingestion reporter, you will need to provide the ```server``` address and the [Wavefront API](https://docs.wavefront.com/wavefront_api.html#generating-an-api-token) ```token```\nHere is an example of the options to provide in a YAML configuration file for direct ingestion, when the server is running on the local machine.\n\n```\n    server: \"http://localhost:8080\"\n    token: \u003cwavefront_api_token\u003e\n    directory: \"/usr/local/foundationdb/logs\"\n    matching: \".*\\\\.xml$\"\n    reporterType: DIRECT\n```\n\n### Using the Graphite Reporter\n\nIf you are running a Graphite server, a Graphite reporter can also be used by specifying the ```graphiteServer``` and ```graphitePort``` corresponding to server.  For example, a Graphite server running on graphite.example.com and listening to port 2003 could be specified on the command line via:\n\n```\n    --type GRAPHITE\n    --graphiteServer graphite.example.com\n    --graphitePort 2003\n    --dir \"/usr/local/foundationdb/logs\"\n    --matching \".*\"\n```\n\n### Using multiple Wavefront endpoints\n\nIn order to send metrics to multiple Wavefront endpoint, you will need to provide a list of endpoints and the [Wavefront API](https://docs.wavefront.com/wavefront_api.html#generating-an-api-token) token. Here is an example of the options to provide in a YAML configuration file. Note that the reporter type must be `DIRECT`. \n```\n    directory: \"/usr/local/foundationdb/logs\"\n    matching: \".*\\\\.xml$\"\n    reporterType: DIRECT\n    endPoints:\n        - endPoint1: token@endPoint1.wavefront.com\n        - endPoint2: token@endPoint2.wavefront.com\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwavefronthq%2Fwavefront-fdb-tailer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwavefronthq%2Fwavefront-fdb-tailer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwavefronthq%2Fwavefront-fdb-tailer/lists"}