{"id":20337316,"url":"https://github.com/expediadotcom/haystack-metrics","last_synced_at":"2025-10-11T00:15:01.543Z","repository":{"id":57719179,"uuid":"102150760","full_name":"ExpediaDotCom/haystack-metrics","owner":"ExpediaDotCom","description":"A library to facilitate collecting metrics in Haystack","archived":false,"fork":false,"pushed_at":"2018-06-22T15:57:41.000Z","size":88,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-07-18T04:12:52.298Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ExpediaDotCom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-01T20:31:03.000Z","updated_at":"2021-02-24T01:29:05.000Z","dependencies_parsed_at":"2022-08-26T09:41:35.699Z","dependency_job_id":null,"html_url":"https://github.com/ExpediaDotCom/haystack-metrics","commit_stats":null,"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/ExpediaDotCom/haystack-metrics","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExpediaDotCom%2Fhaystack-metrics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExpediaDotCom%2Fhaystack-metrics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExpediaDotCom%2Fhaystack-metrics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExpediaDotCom%2Fhaystack-metrics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ExpediaDotCom","download_url":"https://codeload.github.com/ExpediaDotCom/haystack-metrics/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ExpediaDotCom%2Fhaystack-metrics/sbom","scorecard":{"id":47788,"data":{"date":"2025-08-11","repo":{"name":"github.com/ExpediaDotCom/haystack-metrics","commit":"a8d2c4768c6ef4f99ee0a84d6a24be06a49dcef2"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.6,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":7,"reason":"Found 12/17 approved changesets -- score normalized to 7","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-5mg8-w23w-74h3","Warn: Project is vulnerable to: GHSA-7g45-4rm6-3mm3","Warn: Project is vulnerable to: GHSA-mvr2-9pj6-7w5j"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 25 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-14T23:07:50.002Z","repository_id":57719179,"created_at":"2025-08-14T23:07:50.002Z","updated_at":"2025-08-14T23:07:50.002Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279005657,"owners_count":26083941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T21:08:36.812Z","updated_at":"2025-10-11T00:15:01.525Z","avatar_url":"https://github.com/ExpediaDotCom.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Coverage Status](https://coveralls.io/repos/github/ExpediaDotCom/haystack-metrics/badge.svg?branch=master)](https://coveralls.io/github/ExpediaDotCom/haystack-metrics?branch=master)\n[![Build Status](https://travis-ci.org/ExpediaDotCom/haystack-metrics.svg?branch=master)](https://travis-ci.org/ExpediaDotCom/haystack-metrics)\n# haystack-metrics\nThis haystack-metrics module contains code needed by most or all of the other modules in the Haystack code base.\n\n## Metrics\nThe Haystack system is deployed by [Kubernetes](https://en.wikipedia.org/wiki/Kubernetes) and comes with an\n[InfluxDb](https://www.influxdata.com/time-series-platform/influxdb/) database for time series data (TSD).\nOther modules then use Netflix [Servo](https://github.com/Netflix/servo) metrics objects to create two types of metrics:\n1. [Counter](https://github.com/Netflix/servo/blob/master/servo-core/src/main/java/com/netflix/servo/monitor/Counter.java)\nmonitors to track how often an event of interest is occurring, and\n2. [Timer](https://github.com/Netflix/servo/blob/master/servo-core/src/main/java/com/netflix/servo/monitor/Timer.java)\nmonitors to track how much time an event of interest is taking.\n\nGlue code in the [metrics](src/main/java/com/expedia/www/haystack/metrics) package of this module makes it easy to \ncreate Counters and Timers.\n### Usage\n#### Dependencies\nIn the `\u003cproperties\u003e` section of pom.xml put:\n```\n\u003chaystack-metrics-version\u003e...\u003c/haystack-metrics-version\u003e\n```\nYou should of course use the correct version of the haystack-metrics dependency in place of ... above.\n\nIn the `\u003cdependencies\u003e` section of pom.xml put:\n```\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.expedia.www\u003c/groupId\u003e\n    \u003cartifactId\u003ehaystack-metrics\u003c/artifactId\u003e\n    \u003cversion\u003e${haystack-metrics-version}\u003c/version\u003e\n\u003c/dependency\u003e\n```\n#### How to create objects\nIn the examples below, the values of `SUBSYSTEM`, `APPLICATION`, and `CLASS_NAME` should not contain spaces or periods \n(each period or space will be changed to a hyphen).\n##### Subsystem\nAs you will see, creating a Servo object in Haystack requires a \"subsystem\" String, whose value will be something like\n\"pipes\" or \"trends\"; the `SUBSYSTEM` constant below should be defined at a high level in your subsystem code base.\n```\npublic static final String SUBSYSTEM = \"subsystemName\"; // e.g. \"pipes\" or \"trends\"\n```\n##### Application\nApplications are in the subsystem's git repository, and good practice is to store the application name at a high level\nin the application's code hierarchy.\n```\npublic static final String APPLICATION = \"applicationName\";\n```\n##### Class\nCreating a Servo object also requires a \"class\" String, which is often the Java class or Scala object containing the\nobject:\n```\nprivate static final String CLASS_NAME = ClassContainingTheCounter.class.getSimpleName();\n```\nRefactoring or renaming may well lead to changing the name of the Java class or Scala object in which the Servo object\nresides, so it also acceptable to choose a \"class\" String that will never change (and that may not even correspond to\nan actual class in your code base):\n```\nprivate static final String CLASS_NAME = \"JsonSerialization\";\n```\n##### Singleton\nYour Servo objects should be singletons, either as static (Java) or object (Scala) variables. The MetricObjects\nvariable with which you create them can be managed by a Dependency Injection (DI) framework or not, as you see fit;\nnote that if they are managed by a DI framework like [Spring](https://projects.spring.io/spring-framework/)\nyou can choose to let Spring manage the singleton and inject the same Servo object into each object that Spring\ninstantiates. (The examples below show the creation of a new MetricsObject with the creation of each Servo object.) \nServo objects are specified with Identifiers:\n1. Subsystem\n2. Application\n3. Class Name\n4. Metric Name\nIf a Counter or Timer is created twice (that is, its Identifiers match that of an already registered Counter or Timer),\nthen a warning will be logged and the existing object returned by the createAndRegister call. A Timer and a Counter \nwith matching Identifiers is allowed but best avoided.\n#### Counter\n##### Creation\nThe code below is a Java snippet that shows the right way to create a Counter:\n```\nstatic final Counter REQUEST = (new MetricObjects()).createAndRegisterCounter(\n    SUBSYSTEM, APPLICATION, CLASS_NAME, \"REQUEST\");\n```\nBecause the Servo Counter generates a RATE metric, using upper case for the variable name `REQUEST` and the counter name \n`\"REQUEST\"` is recommended because doing so results in an sensibly named complete metric name of `REQUEST_RATE` in\nInfluxDb, as explained in the \"Graphite Bridge\" section of this document. The `sendasrate`configuration controls whether\nrates or counts are sent by the Counters (simple counts are easier to understand and often just as useful as rates).\n##### Usage\nSimply increment the Counter to count:\n```\nREQUEST.increment();\n```\nIt will be reset when its value is reported to InfluxDb.\n#### BasicTimer\n##### Creation\nThe code below is a Java snippet that shows the right way to create a BasicTimer:\n```\nstatic final Timer JSON_SERIALIZATION = (new MetricObjects()).createAndRegisterTimer(\n    SUBSYSTEM, APPLICATION, CLASS_NAME, \"JSON_SERIALIZATION\", TimeUnit.MICROSECONDS);\n```\nThe Servo Timer generates four metrics (GAUGE max, NORMALIZED count, NORMALIZED totalOfSquares, and NORMALIZED\ntotalTime), and while using upper case is again suggested (see the Counter section above), the complete metric names \n(`JSON_SERIALIZATION_GAUGE_min`, `JSON_SERIALIZATION_NORMALIZED_count`, `JSON_SERIALIZATION_NORMALIZED_totalOfSquares`, \nand `JSON_SERIALIZATION_NORMALIZED_totalTime`) are mixed case.\nChoose the appropriate time unit as the last argument:\n* For on-host code, `TimeUnit.MICROSECONDS` is probably appropriate.\n* For network calls, `TimeUnit.MILLISECONDS` may be sufficient.\nThe coarser TimeUnit.MILLISECONDS has less performance impact than the finer TimeUnit.MICROSECONDS and \nTimeUnit.NANOSECONDS; you can read more about this issue\n[here](https://stackoverflow.com/questions/19052316/why-is-system-nanotime-way-slower-in-performance-than-system-currenttimemill)\nand [here](http://stas-blogspot.blogspot.nl/2012/02/what-is-behind-systemnanotime.html).\n##### Usage\nFollow the pattern below (this is for Java; the Scala implementation is similar):\n```\nfinal Stopwatch stopwatch = JSON_SERIALIZATION.start();\ntry {\n    // Do the work being timed\n} finally {\n    stopwatch.stop();\n}\n```\nYou can also do your own timing without using a Stopwatch:\n```\nJSON_SERIALIZATION.record(timeItTookInMs, TimeUnit.MILLISECONDS);\n```\nAgain, the Timer will be reset when its values are reported to InfluxDb.\n#### BucketTimer and StatsTimer\nServo provides counters more complicated than BasicTimer:\n* [BucketTimer](https://netflix.github.io/servo/current/servo-core/docs/javadoc/com/netflix/servo/monitor/BucketTimer.html)\nand \n* [StatsTimer](https://github.com/Netflix/servo/blob/master/servo-core/src/main/java/com/netflix/servo/monitor/StatsTimer.java)\nThey can be created with code very similar to what was given in the BucketTimer section above.\n#### The Main Method\nTo initialize the metrics system, the first line of your main() method should be something like:\n```\n(new MetricPublishing()).start(graphiteConfig);\n```\nwhere graphiteConfig is an implementation of the GraphiteConfig interface declared in this module.\n#### Configuration\nYou will typically have a base.yaml in your resources directory whose contents will include something like:\n```\nhaystack:\n  graphite:\n     prefix: \"haystack\" # using something other than \"haystack\" will require a change in the InfluxDb template\n     host: \"haystack.local\" # set in /etc/hosts per instructions in haystack/deployment module\n     port: 2003 # Graphite port; typically 2003\n     pollintervalseconds: 60\n     queuesize: 10\n     sendasrate: false\n```\n### Graphite Bridge\nThe \"Graphite Bridge\" connects Servo metrics from the application to the Haystack InfluxDb via Graphite \n[plaintext protocol](http://graphite.readthedocs.io/en/latest/feeding-carbon.html#the-plaintext-protocol) messages.\nSuch a message consists of three space-delimited Strings terminated by a newline:\n```\n \u003cmetric path\u003e \u003cmetric value\u003e \u003cmetric timestamp\u003e\\n\n```\nThe `\u003cmetric value\u003e` is a number, and the pieces of `\u003cmetric path\u003e` are traditionally separated by a period.\nNote that the period-delimited pieces contain no metadata; that is, the meanings of each piece are not specified in the\nmessage. This lack of metadata is addressed in [OpenTSDB](http://opentsdb.net) but code to connect Servo metrics to\nInfluxDb via the OpenTSDB protocol does not currently exist in Servo; instead, the bridge uses the Graphite plaintext \nprotocol, and an InfluxDb template (read about them in this \n[README](https://github.com/influxdata/influxdb/blob/master/services/graphite/README.md) file) parses the Graphite plain\ntext message into tags. (You can read about metrics tags \n[here](http://opentsdb.net/docs/build/html/user_guide/query/timeseries.html).)\n\nThis graphite bridge therefore requires a convention to map each metric piece to a tag; this convention is found/used in \nthree places that must agree on the convention:\n1. The template configuration (see the `templates` value in \n[influxdb.yaml](https://github.com/ExpediaDotCom/haystack/blob/master/deployment/k8s/addons/1.6/monitoring/influxdb.yaml))\n2. The code that builds client-side tags (see `getTags()` in \n[MetricObjects.java](src/main/java/com/expedia/www/haystack/metrics/MetricObjects.java))\n3. The code that creates the graphite plain text message from the metric and its client-side tags (see `getName()` in \n[ServoToInfluxDbViaGraphiteNamingConvention.java](src/main/java/com/expedia/www/haystack/metrics/ServoToInfluxDbViaGraphiteNamingConvention.java))\n\nAs a result, the graphite message has the following meaning:\n```\n\u003csystem\u003e.\u003cserver\u003e.\u003csubsystem\u003e.\u003capplication\u003e.\u003cclass\u003e.\u003cVARIABLE_NAME\u003e_\u003cMETRIC_NAME\u003e (for Counter)\n\u003csystem\u003e.\u003cserver\u003e.\u003csubsystem\u003e.\u003capplication\u003e.\u003cclass\u003e.\u003cVARIABLE_NAME\u003e_\u003cMETRIC_NAME\u003e_\u003ctimerStatName\u003e (for Timer)\n```\nwhere:\n* `\u003csystem\u003e` is typically \"haystack\" (this value is controlled by the `haystack.graphite.prefix` configuration)\n* `\u003cserver\u003e` is the host name\n* `\u003csubsystem\u003e` is the value discussed in the \"Subsystem\" section above\n* `\u003capplication\u003e` is the value discussed in the \"Application\" section above\n* `\u003cclass\u003e` is the  value discussed in the \"Class\" section above\n* `\u003cVARIABLE_NAME\u003e_\u003cMETRIC_NAME\u003e` or `\u003cVARIABLE_NAME\u003e_\u003cMETRIC_NAME\u003e_\u003ctimerStatName\u003e` is the complete metric name; see \nthe \"Counter\" and \"BasicTimer\" sections above.\n\n### Releases\n1. Decide what kind of version bump is necessary, based on [Semantic Versioning](http://semver.org/) conventions.\nIn the items below, the version number you select will be referred to as `x.y.z`.\n2. Update the [pom.xml](https://github.com/ExpediaDotCom/haystack-metrics/blob/master/pom.xml), changing the\nversion element to `\u003cversion\u003ex.y.z-SNAPSHOT\u003c/version\u003e`. Note the `-SNAPSHOT` suffix.\n3. Make your code changes, including unit tests. This package requires 100% unit test code coverage for the build to \nsucceed.\n4. Update the [ReleaseNotes.md]((https://github.com/ExpediaDotCom/haystack-metrics/blob/master/ReleaseNotes.md)) file\nwith details of your changes.\n5. Create a pull request with your changes.\n6. Ask for a review of the pull request; when it is approved, the Travis CI build will upload the resulting jar file\nto the [SonaType Staging Repository](https://oss.sonatype.org/#stagingRepositories)\n7. Tag the build with the version number: from a command line, executed in the root directory of the project:\n```\ngit tag x.y.z\ngit push --tags\n```\nThis will cause the jar file to be released to the \n[SonaType Release Repository](https://oss.sonatype.org/#nexus-search;quick~haystack-metrics)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexpediadotcom%2Fhaystack-metrics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fexpediadotcom%2Fhaystack-metrics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fexpediadotcom%2Fhaystack-metrics/lists"}