{"id":19281738,"url":"https://github.com/scality/utapi","last_synced_at":"2026-04-17T02:01:07.939Z","repository":{"id":37250506,"uuid":"65316974","full_name":"scality/utapi","owner":"scality","description":"API for tracking resource utilization and reporting metrics","archived":false,"fork":false,"pushed_at":"2026-04-15T17:37:26.000Z","size":2620,"stargazers_count":14,"open_issues_count":13,"forks_count":3,"subscribers_count":56,"default_branch":"development/8.2","last_synced_at":"2026-04-15T19:28:53.609Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scality.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-08-09T18:02:14.000Z","updated_at":"2026-03-25T18:45:55.000Z","dependencies_parsed_at":"2023-10-10T19:00:12.567Z","dependency_job_id":"60d0e6a5-4fe1-42f5-a00d-d246e72ef7dc","html_url":"https://github.com/scality/utapi","commit_stats":null,"previous_names":[],"tags_count":477,"template":false,"template_full_name":null,"purl":"pkg:github/scality/utapi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scality%2Futapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scality%2Futapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scality%2Futapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scality%2Futapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scality","download_url":"https://codeload.github.com/scality/utapi/tar.gz/refs/heads/development/8.2","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scality%2Futapi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31911846,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T18:22:33.417Z","status":"online","status_checked_at":"2026-04-17T02:00:06.879Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T21:23:58.974Z","updated_at":"2026-04-17T02:01:07.400Z","avatar_url":"https://github.com/scality.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# utapi\n\n![Utapi logo](res/utapi-logo.png)\n\n[![Circle CI][badgepub]](https://circleci.com/gh/scality/utapi)\n\nService Utilization API for tracking resource usage and metrics reporting.\n\n## Design\n\nPlease refer to the [design](/DESIGN.md) for more information.\n\n## Server\n\nTo run the server:\n\n```\nnpm start\n```\n\n## Client\n\nThe module exposes a client, named UtapiClient. Projects can use this client to\npush metrics directly to the underlying datastore (Redis) without the need of an\nextra HTTP request to Utapi.\n\n```js\nconst { UtapiClient } = require('utapi');\n\nconst config = {\n    redis: {\n        host: '127.0.0.1',\n        port: 6379\n    },\n    localCache: {\n        host: '127.0.0.1',\n        port: 6379\n    }\n}\nconst c = new UtapiClient(config);\n\n// The second argument to `pushMetric` is a hexadecimal string Request Unique\n// Identifier used for logging.\nc.pushMetric('createBucket', '3d534b1511e5630e68f0', { bucket: 'demo' });\n\nc.pushMetric('putObject', '3d534b1511e5630e68f0', {\n    bucket: 'demo',\n    newByteLength: 1024,\n    oldByteLength: null,\n});\n\nc.pushMetric('putObject', '3d534b1511e5630e68f0', {\n    bucket: 'demo',\n    newByteLength: 1024,\n    oldByteLength: 256,\n});\n\nc.pushMetric('multiObjectDelete', '3d534b1511e5630e68f0', {\n    bucket: 'demo',\n    byteLength: 1024,\n    numberOfObjects: 999,\n});\n```\n\nIf an error occurs during a `pushMetric` call and the client is unable to record\nmetrics in the underlying datastore, metric data is instead stored in a local\nRedis cache. Utapi attempts to push these cached metrics (every five minutes, by\ndefault) using a component named UtapiReplay. If the `pushMetric` call initiated\nby UtapiReplay fails, the metric is reinserted into the local Redis cache. The\nparticularities of this behavior are configurable. For further information, see\n[design](/DESIGN.md).\n\n## Listing Metrics with Utapi\n\nTo make a successful request to Utapi you would need\n\n 1. [IAM user with a policy giving access to Utapi](#iam-user-with-a-policy-giving-access-to-utapi)\n 2. [Sign request with Auth V4](#signing-request-with-auth-v4)\n\n### IAM user with a policy giving access to Utapi\n\nNote: The examples here use AWS CLI but any AWS SDK is capable of these actions.\n\n**endpoint-url:** This would be `https://\u003chost\u003e:\u003cport\u003e` where your Identity(IAM)\nServer is running.\n\n1. Create an IAM user\n\n    ```\n    aws iam --endpoint-url \u003cendpoint\u003e create-user --user-name \u003cuser-name\u003e\n    ```\n\n2. Create access key for the user\n\n    ```\n    aws iam --endpoint-url \u003cendpoint\u003e create-access-key --user-name \u003cuser-name\u003e\n    ```\n\n3. Define a managed IAM policy\n\n    sample utapi policy\n\n    ```json\n    cat - \u003e utapipolicy.json \u003c\u003cEOF\n    {\n        \"Version\": \"2012-10-17\",\n        \"Statement\": [\n            {\n                \"Sid\": \"utapiMetrics\",\n                \"Action\": [ \"utapi:ListMetrics\" ],\n                \"Effect\": \"Allow\",\n                \"Resource\": [\n                    \"arn:scality:utapi::012345678901:buckets/*\",\n                    \"arn:scality:utapi::012345678901:accounts/*\",\n                    \"arn:scality:utapi::012345678901:users/*\",\n                ]\n            }\n        ]\n    }\n    EOF\n    ```\n\n    In the above sample, the `Resource` property includes a series of Amazon\n    Resource Names (ARNs) used to define which resources the policy applies to.\n    Thus the sample policy applies to a user with an account ID '012345678901',\n    and grants access to metrics at the levels 'buckets', 'accounts', and\n    'users'.\n\n    The account ID of the ARN can also be omitted, allowing any account to\n    access metrics for those resources. As an example, we can extend the above\n    sample policy to allow any account to access metrics at the level 'service':\n\n    ```json\n    ...\n    \"Resource\": [\n        \"arn:scality:utapi::012345678901:buckets/*\",\n        \"arn:scality:utapi::012345678901:accounts/*\",\n        \"arn:scality:utapi::012345678901:users/*\",\n        \"arn:scality:utapi:::service/*\",\n    ]\n    ...\n    ```\n\n    The omission of a metric level denies a user access to all resources at that\n    level. For example, we can allow access to metrics only at the level\n    'buckets':\n\n    ```json\n    ...\n    \"Resource\": [\"arn:scality:utapi::012345678901:buckets/*\"]\n    ...\n    ```\n\n    Further, access may be limited to specific resources within a metric level.\n    For example, we can allow access to metrics only for a bucket  'foo':\n\n    ```json\n    ...\n    \"Resource\": [\"arn:scality:utapi::012345678901:buckets/foo\"]\n    ...\n    ```\n\n    Or allow access to metrics for the bucket 'foo' for any user:\n\n    ```json\n    ...\n    \"Resource\": [\"arn:scality:utapi:::buckets/foo\"]\n    ...\n    ```\n\n4. Create a managed IAM policy\n\n    Once your IAM policy is defined, create the policy using the following\n    command.\n\n    ```\n    aws iam --endpoint-url \u003cendpoint\u003e create-policy --policy-name utapipolicy \\\n     --policy-document file://utapipolicy.json\n    ```\n\n    A sample output of the above command would look like\n\n    ```json\n    {\n        \"Policy\": {\n            \"PolicyName\": \"utapipolicy\",\n            \"CreateDate\": \"2017-06-01T19:31:18.620Z\",\n            \"AttachmentCount\": 0,\n            \"IsAttachable\": true,\n            \"PolicyId\": \"ZXR6A36LTYANPAI7NJ5UV\",\n            \"DefaultVersionId\": \"v1\",\n            \"Path\": \"/\",\n            \"Arn\": \"arn:aws:iam::0123456789012:policy/utapipolicy\",\n            \"UpdateDate\": \"2017-06-01T19:31:18.620Z\"\n        }\n    }\n    ```\n\n    The arn property of the response, which we call `\u003cpolicy arn\u003e`, will be used\n    in the next step to attach the policy to the user.\n\n5. Attach user to the managed policy\n\n    ```\n    aws --endpoint-url \u003cendpoint\u003e iam  attach-user-policy --user-name\n    \u003cuser-name\u003e --policy-arn \u003cpolicy arn\u003e\n    ```\n\nNow the user has access to ListMetrics request in Utapi on all buckets.\n\n### Signing request with Auth V4\n\nThere are two options here.\n\nYou can generate V4 signature using AWS SDKs or the node module aws4. See the\nfollowing urls for reference.\n\n* http://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html\n* http://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html\n* https://github.com/mhart/aws4\n\nYou may also view examples making a request with Auth V4 using various languages\nand AWS SDKs [here](/examples).\n\nAlternatively, you can use a nifty command line tool available in Scality's\nCloudServer.\n\nYou can git clone the CloudServer repo from here\nhttps://github.com/scality/cloudserver and follow the instructions in the README\nto install the dependencies.\n\nIf you have CloudServer running inside a docker container you can docker exec\ninto the CloudServer container as\n\n```\ndocker exec -it \u003ccontainer-id\u003e bash\n```\n\nand then run the command\n\n```\nnode bin/list_metrics\n```\n\nIt will generate the following output listing available options.\n\n```\nUsage: list_metrics [options]\n\n  Options:\n\n    -h, --help                    output usage information\n    -V, --version                 output the version number\n    -a, --access-key \u003caccessKey\u003e  Access key id\n    -k, --secret-key \u003csecretKey\u003e  Secret access key\n    -m, --metric \u003cmetric\u003e         Metric type\n    --buckets \u003cbuckets\u003e           Name of bucket(s) with a comma separator if\n                                  more than one\n    --accounts \u003caccounts\u003e         Account ID(s) with a comma separator if more\n                                  than one\n    --users \u003cusers\u003e               User ID(s) with a comma separator if more than\n                                  one\n    --service \u003cservice\u003e           Name of service\n    -s, --start \u003cstart\u003e           Start of time range\n    -r, --recent                  List metrics including the previous and\n                                  current 15 minute interval\n    -e --end \u003cend\u003e                End of time range\n    -h, --host \u003chost\u003e             Host of the server\n    -p, --port \u003cport\u003e             Port of the server\n    --ssl                         Enable ssl\n    -v, --verbose\n```\n\nAn example call to list metrics for a bucket `demo` to Utapi in a https enabled\ndeployment would be\n\n```\nnode bin/list_metrics --metric buckets --buckets demo --start 1476231300000\n--end 1476233099999 -a myAccessKey -k mySecretKey -h 127.0.0.1 -p 8100 --ssl\n```\n\nBoth start and end times are time expressed as UNIX epoch timestamps **expressed\nin milliseconds**.\n\nKeep in mind, since Utapi metrics are normalized to the nearest 15 min.\ninterval, start time and end time need to be in the specific format as follows.\n\n#### Start time\n\nStart time needs to be normalized to the nearest 15 minute interval with seconds\nand milliseconds set to 0. So valid start timestamps would look something like\n`09:00:00:000`, `09:15:00:000`, `09:30:00:000` and `09:45:00:000`.\n\nFor example\n\nDate: Tue Oct 11 2016 17:35:25 GMT-0700 (PDT)\n\nUnix timestamp (milliseconds): 1476232525320\n\nHere's an example JS method to get a start timestamp\n\n```javascript\nfunction getStartTimestamp(t) {\n    const time = new Date(t);\n    const minutes = time.getMinutes();\n    const timestamp = time.setMinutes((minutes - minutes % 15), 0, 0);\n    return timestamp;\n}\n```\n\nThis would format the start time timestamp to `1476231300000`\n\n#### End time\n\nEnd time needs to be normalized to the nearest 15 minute end interval with\nseconds and milliseconds set to 59 and 999 respectively. So valid end timestamps\nwould look something like `09:14:59:999`, `09:29:59:999`, `09:44:59:999` and\n`09:59:59:999`.\n\nHere's an example JS method to get an end timestamp\n\n```javascript\nfunction getEndTimestamp(t) {\n    const time = new Date(t);\n    const minutes = time.getMinutes();\n    const timestamp = time.setMinutes((minutes - minutes % 15) + 15, 0, -1);\n    return timestamp;\n}\n```\n\nThis would format the end time timestamp to `1476233099999`\n\n## Guidelines\n\nPlease read our coding and workflow guidelines at\n[scality/Guidelines](https://github.com/scality/Guidelines).\n\n### Contributing\n\nIn order to contribute, please follow the\n[Contributing Guidelines](\nhttps://github.com/scality/Guidelines/blob/master/CONTRIBUTING.md).\n\n[badgepub]: http://circleci.com/gh/scality/utapi.svg?style=svg\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscality%2Futapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscality%2Futapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscality%2Futapi/lists"}