{"id":13438175,"url":"https://github.com/performancecopilot/hornet","last_synced_at":"2025-02-27T12:54:48.779Z","repository":{"id":57634422,"uuid":"92884264","full_name":"performancecopilot/hornet","owner":"performancecopilot","description":" A Rust implementation of the PCP instrumentation API ","archived":false,"fork":false,"pushed_at":"2018-01-13T13:06:28.000Z","size":92,"stargazers_count":34,"open_issues_count":2,"forks_count":4,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-10-30T04:29:53.417Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/performancecopilot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-05-30T23:16:17.000Z","updated_at":"2024-04-18T15:31:00.000Z","dependencies_parsed_at":"2022-09-16T04:01:39.160Z","dependency_job_id":null,"html_url":"https://github.com/performancecopilot/hornet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/performancecopilot%2Fhornet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/performancecopilot%2Fhornet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/performancecopilot%2Fhornet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/performancecopilot%2Fhornet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/performancecopilot","download_url":"https://codeload.github.com/performancecopilot/hornet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240959867,"owners_count":19885035,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:03.435Z","updated_at":"2025-02-27T12:54:48.747Z","avatar_url":"https://github.com/performancecopilot.png","language":"Rust","readme":"# hornet [![crates.io badge](https://img.shields.io/crates/v/hornet.svg)](https://crates.io/crates/hornet) [![docs.rs badge](https://docs.rs/hornet/badge.svg)](https://docs.rs/hornet/0.1.0/hornet/) [![Travis CI Build Status](https://travis-ci.org/performancecopilot/hornet.svg?branch=master)](https://travis-ci.org/performancecopilot/hornet) [![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/ccvbo3chne8046vn/branch/master?svg=true)](https://ci.appveyor.com/project/saurvs/hornet-2qtki/branch/master) [![codecov](https://codecov.io/gh/performancecopilot/hornet/branch/master/graph/badge.svg)](https://codecov.io/gh/performancecopilot/hornet)\n\n`hornet` is a Performance Co-Pilot (PCP) Memory Mapped Values (MMV) instrumentation library written in Rust.\n\n**Contents**\n\n* [What is PCP MMV instrumentation?](#what-is-pcp-mmv-instrumentation)\n* [Usage](#usage)\n* [API](#api)\n  * [Singleton Metric](#singleton-metric)\n  * [Instance Metric](#instance-metric)\n  * [Special Metrics](#special-metrics)\n  * [Client](#client)\n  * [Monitoring metrics](#monitoring-metrics)\n* [License](#license)\n\n## What is PCP MMV instrumentation?\n\n[Performance Co-Pilot](http://pcp.io/) is a systems performance analysis framework with a distributed and scalable architecture. It supports a low overhead\nmethod for instrumenting applications called Memory Mapped Values (MMV), in which instrumented processes share part of their virtual memory address space with another monitoring process through a common memory-mapped file. The shared address space contains various performance analysis metrics stored in a structured binary data format called MMV; it's formal spec can be found [here](http://pcp.io/man/man5/mmv.5.html). When processes wish to update their metrics, they simply write certain bytes to the memory mapped file, and the monitoring process reads it at appropriate times. No explicit inter-process communication, synchronization or systems calls are involved.\n\n## Usage\n\n* Add the ```hornet``` dependency to your ```Cargo.toml```\n  ```toml\n  [dependencies]\n  hornet = \"0.1.0\"\n  ```\n\n* Include the ```hornet``` crate in your code and import the following modules\n  ```rust\n  extern crate hornet;\n\n  use hornet::client::Client;\n  use hornet::client::metric::*;\n  ```\n\n## API\n\nThere are essentially two kinds of metrics in `hornet`.\n\n### Singleton Metric\n\nA singleton metric is a metric associated with a primitive value type, a `Unit`, a `Semantics` type, and some metadata. A primitive value can be any one of `i64`, `u64`, `i32`, `u32`, `f64`, `f32`, or `String`, \n\nThe primitive value type of a metric is determined implicitly at *compile-time* by the inital primitive value passed to the metric while creating it. The programmer also needn't worry about reading or writing data of the wrong primitive type from a metric, as the Rust compiler enforces type safety for a metric's primitive value during complilation.\n\nLet's look at creating a simple `i64` metric\n\n  ```rust\n  let mut metric = Metric::new(\n      \"simple\", // metric name\n      1, // inital value of type i64\n      Semantics::Counter,\n      Unit::new().count(Count::One, 1).unwrap(), // unit with a 'count' dimension of power 1\n      \"Short text\", // short description\n      \"Long text\", // long description\n  ).unwrap();\n  ```\n\nIf we want to create an `f64` metric, we simply pass an `f64` inital value instead\n\n  ```rust\n  let mut metric = Metric::new(\n      \"simple_f64\", // metric name\n      1.5, // inital value of type f64\n      Semantics::Instant,\n      Unit::new().count(Time::Sec, 1).unwrap(), // unit with a 'time' dimension of power 1\n      \"Short text\", // short description\n      \"Long text\", // long description\n  ).unwrap();\n  ```\n\nAnd similarly for a `String` metric\n\n  ```rust\n  let mut metric = Metric::new(\n      \"simple_string\", // metric name\n      \"Hello, world!\".to_string(), // inital value of type String\n      Semantics::Discrete,\n      Unit::new().unwrap(), // unit with no dimension\n      \"Short text\", // short description\n      \"Long text\", // long description\n  ).unwrap();\n  ```\n\nThe detailed API on singleton metrics can be found [here](https://docs.rs/hornet/0.1.0/hornet/client/metric/struct.Metric.html).\n\n### Instance Metric\n\nAn instance metric is similar to a singleton metric in that it is also associated with a primitive valye type, `Unit`, and `Semantics`,\nbut additionally also holds multiple independent primitive values of the same type. The same type inference rules also hold for instance metrics - the type\nof the inital value determines the type of the instance metric.\n\nBefore we can create an instance metric, we need to create what's called an\n*instance domain*. An instance domain is a set of `String` values that act\nas unique identifiers for the multiple independent values of an instance metric. Why have a separate object for this purpose? So that we can reuse the same identifiers as a \"domain\" for several different but related instance metrics. An example will clear this up.\n\nSuppose we are modeling the fictional [Acme Corporation factory](https://en.wikipedia.org/wiki/Acme_Corporation). Let's assume we have three items that can be manufactured - Anvils, Rockets, and Giant Rubber Bands. Each item is associated with a \"count\" metric of how many copies have been manufactured so far, and a \"time\" metric of how much time has been spent manufacturing each item. We can create instance metrics like so\n\n  ```rust\n  /* instance domain */\n  let indom = Indom::new(\n      \u0026[\"Anvils\", \"Rockets\", \"Giant_Rubber_Bands\"],\n      \"Acme products\", // short description\n      \"Most popular products produced by the Acme Corporation\" // long description\n  ).unwrap();\n    \n  /* two instance metrics */\n\n  let mut counts = InstanceMetric::new(\n      \u0026indom,\n      \"products.count\", // instance metric name\n      0, // inital value of type i64\n      Semantics::Counter,\n      Unit::new().count(Count::One, 1).unwrap(),\n      \"Acme factory product throughput\",\n      \"Monotonic increasing counter of products produced in the Acme Corporation factory since starting the Acme production application.\"\n  ).unwrap();\n\n  let mut times = InstanceMetric::new(\n      \u0026indom,\n      \"products.time\",  // instance metric name\n      0.0, // inital value of type f64\n      Semantics::Instance,\n      Unit::new().time(Time::Sec, 1).unwrap(),\n      \"Time spent producing products\",\n      \"Machine time spent producing Acme Corporation products.\"\n  ).unwrap();\n\n  ```\n\nHere, our `indom` contains three identifiers - `Anvils`, `Rockets` and `Giant_Rubber_Bands`.\nWe've created two instance metrics - `counts` of type `i64` and `times` of type `f64` with relevant units and semantics.\n\nThe detailed API on instance metrics can be found [here](https://docs.rs/hornet/0.1.0/hornet/client/metric/struct.InstanceMetric.html).\n\n### Updating metrics\n\nSo far we've seen how to create metrics with various attributes. Updating their primitive values is pretty simple.\n\nFor singleton metrics, the `val(\u0026self) -\u003e \u0026T` method returns a reference to the underlying value, and the\n`set_val(\u0026mut self, new_val: T) -\u003e io::Result\u003c()\u003e` method updates the underlying value and\nwrites to the memory mapped file. The arguments and return values for these methods\nare generic over the different primitive types for a metric, and hence are completely type safe.\n\nFor instance metrics, the `val(\u0026self, instance: \u0026str) -\u003e Option\u003c\u0026T\u003e` method returns a reference to the primitive value for the given instance identifier, if it exists. The\n`set_val(\u0026mut self, instance: \u0026str, new_val: T) -\u003e Option\u003cio::Result\u003c()\u003e\u003e` method updates the primitive value for the given instance identifier, if it exists. These methods are similarly\ngeneric over primitive value types.\n\n## Special metrics\n\nSingleton metrics and instance metrics are powerful and general enough to be used for a wide variety of performance analysis needs. However, for many common applications, simpler metric interfaces would be more appropriate and easy to use. Hence `hornet` includes 6 high-level metrics that are built on top of singleton and instance metrics, and they offer a more specialized and simpler API.\n\n#### Counter\n\nA `Counter` is a singleton metric of type `u64`, `Counter` semantics, and unit of 1 count dimension. It implements the following methods: `up` to increment by one, `inc` to increment\nby a delta, `reset` to set count to the inital count, and `val` to return the current count.\n\n  ```rust\n  let mut c = Counter::new(\n      \"counter\", // name\n      1, // inital value\n      \"\", \"\" // short and long description strings\n  ).unwrap();\n\n  c.up(); // 2\n  c.inc(3); // 5\n  c.reset(); // 1\n\n  let count = c.val(); // 1\n  ```\n\nThe [CountVector](https://docs.rs/hornet/0.1.0/hornet/client/metric/struct.CountVector.html) is the instance metric version of the `Counter`. It holds multiple counts each associated with a `String` identifier.\n\n#### Gauge\n\nA `Gauge` is a singleton metric of type `f64`, `Instant` semantics, and unit of 1 count dimension. It implements the following methods: `inc` to increment the gauge by a delta, `dec` to decrement the gauge by a delta, `set` to set the gauge to an arbritrary value, and `val` which returns the current value of the gauge.\n\n  ```rust\n  let mut gauge = Gauge::new(\"gauge\", 1.5, \"\", \"\").unwrap();\n  \n  gauge.set(3.0).unwrap(); // 3.0\n  gauge.inc(3.0).unwrap(); // 6.0\n  gauge.dec(1.5).unwrap(); // 4.5\n  gauge.reset().unwrap();  // 1.5\n  ```\n\nThe [GaugeVector](https://docs.rs/hornet/0.1.0/hornet/client/metric/struct.GaugeVector.html) is the instance metric version of the `Gauge`. It holds multiple gauge values each associated with an identifier.\n\n#### Timer\n\nA `Timer` is a singleton metric of type `i64`, `Instant` semantics, and a user specified time unit. It implements the following methods: `start` starts the timer by recording the current time, `stop` stops the timer by recording the current time\nand returns the elapsed time since the last `start`, and `elapsed` returns the\ntotal time elapsed so far between all start and stop pairs.\n\n  ```rust\n  let mut timer = Timer::new(\"timer\", Time::MSec, \"\", \"\").unwrap();\n\n  timer.start().unwrap();\n  let e1 = timer.stop().unwrap();\n\n  timer.start().unwrap();\n  let e2 = timer.stop().unwrap();\n\n  let elapsed = timer.elapsed(); // = e1 + e2\n  ```\n\n#### Histogram\n\nA `Histogram` is a high dynamic range (HDR) histogram metric which records `u64` data points and exports various statistics about the data. It is implemented using an instance metric of `f64` type and `Instance` semantics. The `Histogram` metric is infact essentially a wrapper around the `Histogram` object from the [hdrsample](https://github.com/jonhoo/hdrsample) crate, and it exports the maximum, minimum, mean and standard deviation statistics to the MMV file.\n\n  ```rust\n  let low = 1;\n  let high = 100;\n  let sigfig = 5;\n\n  let mut hist = Histogram::new(\n      \"histogram\",\n      low,\n      high,\n      sigfig,\n      Unit::new().count(Count::One, 1).unwrap(),\n      \"Simple histogram example\", \"\"\n  ).unwrap();\n\n  let range = Range::new(low, high);\n  let mut thread_rng = thread_rng();\n\n  for _ in 0..100 {\n      hist.record(range.ind_sample(\u0026mut thread_rng)).unwrap();\n  }\n  ```\n\nMuch of the `Histogram` [API](https://docs.rs/hornet/0.1.0/hornet/client/metric/struct.Histogram.html) is largely similar to the [hdrsample API](https://docs.rs/hdrsample/6.0.1/hdrsample/struct.Histogram.html).\n\n### Client\n\nIn order to export our metrics to a memory mapped file, we must first create a `Client`\n\n  ```rust\n  let client = Client::new(\"client\").unwrap(); // MMV file will be named 'client'\n  ```\n\nNow to export metrics, we simply call `export`\n\n  ```rust\n  client.export(\u0026mut [\u0026mut metric1, \u0026mut metric2, \u0026mut metric3]);\n  ```\n\nIf you have a valid PCP installation, the `Client` writes the MMV file to `$PCP_TMP_DIR/mmv/`, and otherwise it writes it to `/tmp/mmv/`.\n\nAfter metrics are exported through a `Client`, all updates to their primitive values will show up in the MMV file.\n\n### Monitoring metrics\n\nWith a valid PCP installation on a machine, metrics can be monitored externally by using the follwing command\n  ```bash\n  $ pminfo -f mmv._name_\n  ```\nwhere `_name_` is the name passed to `Client` while creating it.\n\nAnother way to inspect metrics externally is to dump the contents of the MMV file itself. This can be done using a command line tool called `mmvdump` included in `hornet`. After issuing `cargo build` from within the project directory, `mmvdump` can be found built under `target/debug/`.\n\nUsage of `mmvdump` is pretty straightforward\n\n  ```rust\n  $ ./mmvdump simple.mmv\n\n  Version    = 1\n  Generated  = 1468770536\n  TOC count  = 3\n  Cluster    = 127\n  Process    = 29956\n  Flags      = process (0x2)\n\n  TOC[0]: toc offset 40, metrics offset 88 (1 entries)\n    [725/88] simple.counter\n        type=Int32 (0x0), sem=counter (0x1), pad=0x0\n        unit=count (0x100000)\n        (no indom)\n        shorttext=A Simple Metric\n        longtext=This is a simple counter metric to demonstrate the hornet API\n\n  TOC[1]: toc offset 56, values offset 192 (1 entries)\n    [725/192] simple.counter = 42\n\n  TOC[2]: toc offset 72, strings offset 224 (2 entries)\n    [1/224] A Simple Metric\n    [2/480] This is a simple counter metric to demonstrate the hornet API\n  ```\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally\nsubmitted for inclusion in the work by you, as defined in the Apache-2.0\nlicense, shall be dual licensed as above, without any additional terms or\nconditions.\n","funding_links":[],"categories":["Development tools","开发工具","开发工具 Development tools"],"sub_categories":["Profiling","剖析","分析 Profiling"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fperformancecopilot%2Fhornet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fperformancecopilot%2Fhornet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fperformancecopilot%2Fhornet/lists"}