{"id":21893066,"url":"https://github.com/peekjef72/httpapi_exporter","last_synced_at":"2025-04-15T15:08:36.232Z","repository":{"id":205650654,"uuid":"695935033","full_name":"peekjef72/httpapi_exporter","owner":"peekjef72","description":"generic http REST API exporter","archived":false,"fork":false,"pushed_at":"2025-03-30T15:25:01.000Z","size":2528,"stargazers_count":8,"open_issues_count":3,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-15T15:08:20.851Z","etag":null,"topics":["arubacx","exporter","hp3par","httpapi","json","netscaler","prometheus","veeam"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peekjef72.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-24T16:57:09.000Z","updated_at":"2025-04-07T13:10:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"1c479df8-0033-40ea-93b7-eba6301b455b","html_url":"https://github.com/peekjef72/httpapi_exporter","commit_stats":{"total_commits":12,"total_committers":2,"mean_commits":6.0,"dds":0.08333333333333337,"last_synced_commit":"adebc89cd0c0bd69510288ad746e4bda3e1b975f"},"previous_names":["peekjef72/httpapi_exporter"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peekjef72%2Fhttpapi_exporter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peekjef72%2Fhttpapi_exporter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peekjef72%2Fhttpapi_exporter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peekjef72%2Fhttpapi_exporter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peekjef72","download_url":"https://codeload.github.com/peekjef72/httpapi_exporter/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249094932,"owners_count":21211837,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arubacx","exporter","hp3par","httpapi","json","netscaler","prometheus","veeam"],"created_at":"2024-11-28T13:01:23.130Z","updated_at":"2025-04-15T15:08:36.225Z","avatar_url":"https://github.com/peekjef72.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cmeta name=\"google-site-verification\" content=\"W1W5S9kNhNL2ZcTEZJ6lMZpAGeNnka6I3iIVFhiUO-I\" /\u003e\n\n# Prometheus HTTPAPI Exporter\n\nThis exporter wants to be a generic REST API exporter. That's mean it can login, then makes requests to urls to collect metrics and performs transformations on values and finally returns metrics in prometheus format.\n\nNothing is hard coded in the exporter. That why it is a generic exporter.\n\nThe queried url can return data in several format:\n\n- json (default parser)\n- yaml\n- xml\n- raw text (parse as text-lines)\n- prometheus openmetrics\n\nAs examples 4 configurations for exporters are provided (see contribs):\n\n- [hp3par_exporter](contribs/hp3par/README.md)\n- [veeam_exporter](contribs/veeam/README.md)\n- [netscaler_exporter](contribs/netscaler/README.md)\n- [aruba_exporter](contribs/arubacx/README.md)\n\n# Build\n\n## use promu tool\n\n```shell\ngo install github.com/prometheus/promu@latest\ngo install github.com/peekjef72/passwd_encrypt@latest\n\n$GOBIN/promu build\n```\n\nor simply make.\n\n```shell\nmake\n```\n\nthis will build the exporter and a tool to crypt/decrypt ciphertext with a shared passphrase.\n\n## Usage\n\n```text\nusage: httpapi_exporter [\u003cflags\u003e]\n\n\nFlags:\n  -h, --[no-]help            Show context-sensitive help (also try --help-long and --help-man).\n      --web.listen-address=\":9321\"\n                             The address to listen on for HTTP requests.\n      --web.telemetry-path=\"/metrics\"\n                             Path under which to expose collector's internal metrics.\n  -c, --config.file=\"config/config.yml\"\n                             Exporter configuration file.\n  -n, --[no-]dry-run         Only check exporter configuration file and exit.\n  -t, --target=TARGET        In dry-run mode specify the target name, else ignored.\n  -a, --auth.key=AUTH.KEY    In dry-run mode specify the auth_key to use, else ignored.\n  -o, --collector=COLLECTOR  Specify the collector name restriction to collect, replace the collector_names set for each target.\n      --log.level=info       Only log messages with the given severity or above. One of: [debug, info, warn, error]\n      --log.format=logfmt    Output format of log messages. One of: [logfmt, json]\n  -V, --[no-]version         Show application version.\n```\n\n## Loging level\n\nYou can change the log.level online by sending a signal USR2 to the process. It will increase and cycle into levels each time a signal is received.\n\n```shell\nkill -USR2 pid\n```\n\nUsefull if something is wrong and you want to have detailled log only for a small interval.\n\nYou can also set the loglevel using API endpoint /loglevel:\n\n- GET /loglevel : to retrieve the current level\n- POST /loglevel : to cycle and increase the current loglevel\n- POST /loglevel/\\\u003clevel\\\u003e : to set level to \\\u003clevel\\\u003e\n\n## Reload\n\nYou can tell the exporter to reload its configuration by sending a signal HUP to the process or send a POST request to /reload endpoint.\n\n## Exporter configuration\n\nExporter requires configuration to works:\n\n- globals parameters\n- collectors\n- targets\n- authentication definitions\n  \n  see [config.md](doc/config.md) documentation\n\n## Exporter http server\n\nThe exporter http server has a default landing page that permit to access:\n\n- **/health** : a simple heartbeat page that return \"OK\" if exporter is UP\n- **/configuration**: expose defined configuration of the exporter\n- **/targets**: expose all known targets (locally defined or dynamically defined). Password are masked.\n- **/status**: expose exporter version, process start time\n- **/profiling**: expose exporter debug/profiling metrics\n- **/httpapi_exporter_metrics**: exporter internal prometheus metrics\n- **/help**: help on github.\n- **/metrics**: expose target's metrics. Require a target parameter with valid value.\n- **/loglevel**: GET exposes exporter current log level. POST /loglevel increases by one the current level (cycling). POST /loglevel/[level] set the new [level].\n- **/reload**: method POST only: tells the exporter to reload the configuration.\n\n## exporter metrics access\n\nParameters to scrape a target:\n\n- **target**: `\u003clocally_defined_target\u003e` or `\u003cscheme://[user:password@]host:port\u003e` (dynamic target)\n- **auth_key**: the shared secret key used to decrypt encrypted password set in authentication config.\n- **auth_name**: the name of authentication config to use to access to a target.\n- if target is not defined locally (so it is dynamically defined), you can set the authentication parameters to use for that target using those specified in the auth_name config.\n- **model**: the name of model target to use to build dynamic target. If not specified it looks for target named \"default\". This parameter is used only at the first call for the dynamic target creation.\n\n**examples**:\n\n1. `/metrics?target=mytarget` scrapes the target `mytarget` without any parameter; It must be fully defined in the exporter configuration files; it has either no authentication or password is not encrypted.\n2. `/metrics?auth_key=\u003ccipherkey\u003e\u0026target=mytarget2` scrapes the target `mytarget2` that is fully defined in the exporter configuration files, and has a password that is encrypted and can be decrypted with the cipherkey.\n3. `/metrics?auth_key=\u003ccipherkey\u003e\u0026target=\u003chttps://myhost.domain.name:port\u003e\u0026auth_name=\u003cauth_name\u003e` define a dynamic target that is reachable at url `https://myhost.domain.name:port`, using the authentication parameters defined by `\u003cauth_name\u003e` and \"default\" model, then scrapes it. This target was not initially defined in the exporter configuration files, and only exists until the exporter is running. `\u003cauth_name\u003e` has a encrypted password and so requires `\u003ccipherkey\u003e` to decipher it.\n4. `/metrics?auth_key=\u003ccipherkey\u003e\u0026target=\u003chttps://myhost.domain.name:port\u003e\u0026auth_name=\u003cauth_name\u003e\u0026model=mytarget` same than previous example but the dynamic target creation use \"mytarget\" model instead of default.\n\n## Authentication\n\nMost of the time the access to a http api requires an authentication. It is the case for the 3 contribs (hp3par, veeam, netscaler).\nThe exporter allows you 2 modes:\n\n- to define the authentication parameters statically for each target.\n- to define a dictionnary (map) of authentications and then to use it a target definition, or even to set it in the prometheus query.\n\nThe auth parameters are:\n\n- mode: should be:\n  - basic: use http basic authentication: the \"user\" and \"password\" values will be sent in the http header request\n  - token: use bearer token to authenticate\n  - script: user, password will be used in the login script to access the api.\n- user\n- password\n- token\n\nThe user, password and token values can be raw strings or retrive the values from environment variables. To use env var the value must be prefixed by \"$env:\" followed by the name of the variable.\n\n```yml\n  # use this auth_config to authenticate via env vars\n  default:\n    mode: script\n    user: $env:VEEAM_EXPORTER_USER\n    password: $env:VEEAM_EXPORTER_PASSWD\n```\n\n## password encryption\n\nIf you don't want to write the users' password in clear text in config file (targets files on the exporter), you can encrypt them with a shared password.\n\nHow it works:\n\n- choose a shared password (passphrase) of 16 24 or 32 bytes length and store it your in your favorite password keeper (keepass for me).\n- use passwd_encrypt tool:\n\n  ```bash\n  ./passwd_encrypt \n  give the key: must be 16 24 or 32 bytes long\n  enter key: 0123456789abcdef \n  enter password: mypassword\n  Encrypting...\n  Encrypted message hex: CsG1r/o52tjX6zZH+uHHbQx97BaHTnayaGNP0tcTHLGpt5lMesw=\n  $\n  ```\n\n- set the user password in the target file or in auth_configs part:\n\n    ```yaml\n    name: hp3parhost\n    scheme: https\n    host: \"1.2.3.4\"\n    port: 8080\n    baseUrl: /api/v1\n    auth_config:\n      # mode: basic(default)|token|[anything else:=\u003e user defined login script]\n      user: \u003cuser\u003e\n      # password: \"/encrypted/base64_encrypted_password_by_passwd_crypt_cmd\"\n      password: /encrypted/CsG1r/o52tjX6zZH+uHHbQx97BaHTnayaGNP0tcTHLGpt5lMesw=\n    ```\n\n    or\n\n    ```yaml\n    auth_configs:\n      \u003cauth_name\u003e:\n        # mode: basic|token|[anything else:=\u003e user defined login script]\n        mode: \u003cmode\u003e\n        user: \u003cuser\u003e\n        # password: \"/encrypted/base64_encrypted_password_by_passwd_crypt_cmd\"\n        password: /encrypted/CsG1r/o52tjX6zZH+uHHbQx97BaHTnayaGNP0tcTHLGpt5lMesw=\n\n    ```\n\n- set the shared passphrase in prometheus config (either job or node file)\n\n  - prometheus jobs with target files:\n\n    ```yaml\n    #--------- Start prometheus hp3par exporter  ---------#\n    - job_name: \"hp3par\"\n        metrics_path: /metrics\n        file_sd_configs:\n        - files: [ \"/etc/prometheus/hp3par_nodes/*.yml\" ]\n        relabel_configs:\n        - source_labels: [__address__]\n            target_label: __param_target\n        - source_labels: [__tmp_source_host]\n            target_label: __address__\n\n    #--------- End prometheus hp3par exporter ---------#\n    ```\n\n    ```yaml\n    - targets: [ \"hp3par_node_1\" ]\n    labels:\n        __tmp_source_host: \"hp3par_exporter_host.domain.name:9321\"\n    # if you have activated password encrypted passphrass\n        __param_auth_key: 0123456789abcdef\n        host: \"hp3par_node_1_fullqualified.domain.name\"\n        # custom labels…\n        environment: \"DEV\"\n    ```\n\n# custom functions for templating\n\n(**still incomplete**)\n\nGo template is used to manipulate data. So templates inherits from go functions and [sprig](https://masterminds.github.io/sprig/) v3 functions.\nBecause the exporter uses most of the time data of type \"any (interface{})\" some of the sprig functions failed.\nHere is the list of exporter functions:\n\nname| usage | e.g. |\n--|---|---|\nexporterDecryptPass | | |\nexporterGet [varmap] [keyname] | get the keyname from the map. Like sprig/get function but accepts data of type map[any]any | exporterGet .svclb .svc |\nexporterSet | | |\nexporterKeys | | |\nexporterValues | | |\nexporterToRawJson | | |\n\u0026nsbp; | | |\nlookupAddr [varstring] | obtain hostname from string representing an ip address ; like sprig/getHostByName but for string ip | lookupAddr .node.ipaddress |\nconvertToBytes value unit | convert the value contained in variable to bytes according to the unit string specified: \u003cul\u003e\u003cli\u003e\"kilobyte\" or \"Kb\" multiply by 1024 \u003cli\u003e\"megabyte\" or \"Mb\" multiply by 1024 * 1024\u003cli\u003e\"gigabyte\" or \"Gb\"multiply by 1024 * 1024 * 1024\u003c/ul\u003e | '{{ convertToBytes .result.totalMiB \"Mb\" }}' |\nconvertBoolToInt value | convert value that may contain a boolean to 0\u0026#124;1 representation. Value can be of any type. If something is \u003cul\u003e\u003cli\u003elike int or float and different from 0 is 1 else 0\u003cli\u003estring and is lower case 'true' or 'yes' or 'ok' is 1 else 0\u003cli\u003elike map or array and length \u003e0 then 1 or 0\u003c/ul\u003e| with {\"proc\": {\"loopCrashing\": \"true\",...}}\u003cbr\u003e =\u003e '{{ convertBoolToInt .proc.loopCrashing }}\u003cbr\u003e =\u003e 1' |\ngetHeader [varmap] | | |\nLEN [var] | obtain the len of the var. works like sprig/len but accepts data of type any. | |\nexporterRegexExtract [regexp var] [search var] : []string | obtain the list of extracted elements from regexp on search string or nil if not found | extract value from line as group 1 of regexp: \u003cbr\u003e res: \"{{ index  (exporterRegexExtract \"^status:\\s(.+)$\" \"status:OK\") 1 }}\" |\n\n## boolean checks\n\nname| usage | e.g. |\n---|---|---|\nEQ [var1] [var2] | check equality for 2 variables; accepts any type of data; meaning that the second will be converted to the type of the first | EQ .val \"2\" |\nNE [var1] [var2] | not equal | NE 2 .val |\nGE [var1] [var2] | greater equal | |\nGT [var1] [var2] | greater than | |\nLE [var1] [var2] | less equal | |\nLT [var1] [var2] | less than | |\nexists [var1] | return boolean if variable exists| exists .config.cluster.node |\nexporterHasKey [var] [key] | check if variable is a map and has a key | exporterHasKey .config \"cluster\" |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeekjef72%2Fhttpapi_exporter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeekjef72%2Fhttpapi_exporter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeekjef72%2Fhttpapi_exporter/lists"}