{"id":18612501,"url":"https://github.com/framna-sysadmin/telegraf-aws-cost-usage","last_synced_at":"2025-11-03T00:30:25.578Z","repository":{"id":138301137,"uuid":"94326414","full_name":"move-sysadmin/telegraf-aws-cost-usage","owner":"move-sysadmin","description":"Amazon Web Services Cost and Usage data extracted to Telegraf","archived":false,"fork":false,"pushed_at":"2017-06-14T12:03:30.000Z","size":20,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-12-27T02:21:33.089Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/move-sysadmin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-06-14T12:02:24.000Z","updated_at":"2023-03-02T14:57:11.000Z","dependencies_parsed_at":"2024-11-22T02:16:28.287Z","dependency_job_id":null,"html_url":"https://github.com/move-sysadmin/telegraf-aws-cost-usage","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/move-sysadmin%2Ftelegraf-aws-cost-usage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/move-sysadmin%2Ftelegraf-aws-cost-usage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/move-sysadmin%2Ftelegraf-aws-cost-usage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/move-sysadmin%2Ftelegraf-aws-cost-usage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/move-sysadmin","download_url":"https://codeload.github.com/move-sysadmin/telegraf-aws-cost-usage/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239403662,"owners_count":19632627,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T03:17:38.171Z","updated_at":"2025-11-03T00:30:25.497Z","avatar_url":"https://github.com/move-sysadmin.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AWS Cost and Usage for Telegraf\n\nThis repository contains PHP scripts extracting Amazon Web Services Cost and Usage data\nto be feeded into Telegraf\n\n\n## Usage\n\n```\n./extract_report.php -c CONFIG_FILE [-m TARGET_MONTH]\n\nRequired option:\n    -c   CONFIG_FILE    define the config file to load\n                        must contain all variables as in the config_example.php\n\nOptional option:\n    -m   TARGET_MONTH   define the report month to extract\n                        value can be either 'current' or 'previous'\n                        default value is 'current'\n```\n\n\n## Restrictions\n\n- The scripts do NOT download reports from Amazon S3 but expect them to be available locally,\nfor example through [s3fs](https://github.com/s3fs-fuse/s3fs-fuse)\n- Reports are expected to use GZIP compression\n- Only [influx output format](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#influx) is supported\n- Only a subset of each report line is sent to Telegraf  \n  The choice of those fields was made towards the creation of metrics regarding data transfer and overall price per account\n\n\n\n## Reports overview\n\nThe AWS Cost and Usage reports can be managed from the\n[Amazon console](https://console.aws.amazon.com/billing/home#/reports)  \nAll reports are stored in an Amazon S3 bucket.  \nEach individual report contains the full data for the current billing month at the time of generation.  \nThe structure of the bucket folders and files, and the format of the CSV files are defined in the\n[Amazon docs](http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-reports.html#enhanced-organization)\n\nExtra notes:\n- Amazon seem to create a new report twice per day (~6am and ~6pm)\n- At the end of a month, the final days' data is missing from the report.  \nThey seem to be added after ~5 days in the following month.\n\n\n## Recommended setup\n\nGiven the way the reports are generated by Amazon, it is recommended to feed the data to Telegraf\nthrough the [tail input plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/tail)\nand 2 cronjobs.  \nOne twice daily import for the current billing month.  \nOne monthly import for the previous billing month (to fix the last few days).\n\nTelegraf tail input config\n```\n[[inputs.tail]]\n  files = [ \"/var/tmp/aws_cost_usage.out\" ]\n  data_format = \"influx\"\n```\n\nCronjobs config\n```\n0 7,19 * * * extract_report.php -c myconfig.php -m current \u003e /var/tmp/aws_cost_usage.out 2\u003e\u003e /var/log/aws_cost_usage.log\n0 3 7 * * extract_report.php -c myconfig.php -m previous \u003e /var/tmp/aws_cost_usage.out 2\u003e\u003e /var/log/aws_cost_usage.log\n```\n\n\n## Similar project\n\n- [awsbill2graphite](https://github.com/danslimmon/awsbill2graphite)  \nUsing Python  \nDownload reports from Amazon S3  \nSend different metrics directly to Graphite\n\n\n## TODO\n\n- Use compression defined in Manifest file to decompress reports\n- add `-h` cli argument to print usage\n- validate that the config file contains all required variables\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fframna-sysadmin%2Ftelegraf-aws-cost-usage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fframna-sysadmin%2Ftelegraf-aws-cost-usage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fframna-sysadmin%2Ftelegraf-aws-cost-usage/lists"}