{"id":22671475,"url":"https://github.com/cloudforet-io/plugin-http-file-cost-datasource","last_synced_at":"2025-08-02T15:38:08.136Z","repository":{"id":65640332,"uuid":"587957842","full_name":"cloudforet-io/plugin-http-file-cost-datasource","owner":"cloudforet-io","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-18T12:21:17.000Z","size":199,"stargazers_count":1,"open_issues_count":1,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-26T07:04:26.569Z","etag":null,"topics":["cost-analysis","data-source","plugin"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudforet-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-01-12T01:28:43.000Z","updated_at":"2024-10-18T12:21:07.000Z","dependencies_parsed_at":"2023-04-08T09:20:39.172Z","dependency_job_id":"2c8fb8d8-b69c-4f57-9fad-82940c80dc34","html_url":"https://github.com/cloudforet-io/plugin-http-file-cost-datasource","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudforet-io%2Fplugin-http-file-cost-datasource","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudforet-io%2Fplugin-http-file-cost-datasource/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudforet-io%2Fplugin-http-file-cost-datasource/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudforet-io%2Fplugin-http-file-cost-datasource/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudforet-io","download_url":"https://codeload.github.com/cloudforet-io/plugin-http-file-cost-datasource/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248565279,"owners_count":21125447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cost-analysis","data-source","plugin"],"created_at":"2024-12-09T16:15:10.977Z","updated_at":"2025-04-12T12:22:42.349Z","avatar_url":"https://github.com/cloudforet-io.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# plugin-http-file-cost-datasource\n\n* Plugin for collecting cost data from **CSV file**\n\n---\n\n## 1) Overview\nㅡ\nThis plugin is a plugin that collects cost data from a CSV file.  \nThe CSV file must be in the format specified in the [2) CSV format](#2-csv-format) section.  \n\nThe CSV file must be located on the web server,  \nand the URL must be specified in the [3) options of the plugin](#3-options-of-plugin).  \n\nIf you have completed understanding the steps 2) and 3),  \nyou can check the actual usage in [4) How to use](#4-how-to-use).\n\n## 2) CSV format\n\n![img.png](examples/img.png)\n\n* The above is an example of a csv file, and the fields that exist in the csv must exist.  \n* Here is a list of available fields. Of these, `cost` fields are required field.  \n* The required date fields are:  \n  * If there is a `billed_date` (ex. \"2023-09-01\") field, it works even if there are no `year` and `month` fields.\n  * This works if there is no `billed_date` field and there are `year`, `month`, and `day` fields.\n  * If there is no `billed_date` field and there are `year` and `month`, `day` is applied as 1 day.\n  * Does not work except for the above cases.\n\n\u003cbr\u003e\n\n* **cost (required)**\n* **billed_date (required)**\n* **year (required)**\n* **month (required)**\n* **day (optional)**\n* usage_quantity\n* usage_type\n* provider\n* region_code\n* product\n\n## 3) Options of plugin\n\n* The following options are available for the plugin.\n* The options are specified in the form of a YAML file.\n* You can find out how yaml is used in [4) How to use](#4-how-to-use) section.\n* An example using all options is shown below.\n  ```yaml\n  # update_data_source_options.yml\n  ---\n  options:\n    base_url:\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/cost_example.csv\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/custom_cost_example.csv\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/example_with_billed_at.csv\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/examples_of_different_headers.csv\n    field_mapper:\n      cost: TotalCost\n      currency: CurrencyCode\n      day: Day\n      month: Month\n      provider: Provider\n      usage_quantity: UsageQuantity\n      usage_type: UsageType\n      year: Year\n    default_vars:\n      currency: KRW\n    billed_at: UsageStartDate\n  ```\n\n**base_url (required)**\n\n* The URL of the CSV file to be used.\n* The URL must be a list type.\n\n```yaml\n---\noptions:\n  base_url:\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/cost_example.csv\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/custom_cost_example.csv\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/example_with_billed_at.csv\n    - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/examples_of_different_headers.csv\n```\n\n**field_mapper (optional)**\n\n* The field name of the CSV file to be used.\n* If you do not specify this option, the [default column](#2-csv-format) is used.\n* As you can see in [examples_of_different_headers.csv](examples/examples_of_different_headers.csv), if the column names\n  are different, you can use these options to map them.\n\n```yaml\n---\noptions:\n  field_mapper:\n    cost: TotalCost\n    currency: CurrencyCode\n    day: Day\n    month: Month\n    provider: Provider\n    usage_quantity: UsageQuantity\n    usage_type: UsageType\n    year: Year\n```\n\n**default_vars (optional)**\n\n* If you want to set a default value for a field, you can use this option.\n* If you do not specify this option, the [default column](#2-csv-format) is used.\n* As you can see in [example_of_default_vars.csv](examples/example_of_default_vars.csv), This can be used if you want to\n  default all currencies to KRW.\n\n```yaml\n---\noptions:\n  default_vars:\n    currency: KRW\n```\n\n## 4) How to use (Deprecated)\n\nIn order to use the plugin, how to use [spacectl CLI tools](https://github.com/cloudforet-io/spacectl) must be preceded.\n\n1. Check if the plugin you want to use from the marketplace exists.\n\n```shell\n$ spacectl list repository.Plugin -p service_type=cost_analysis.DataSource --minimal\n\nplugin_id                        | name                                | image                                     | state   | service_type             | registry_type\n----------------------------------+-------------------------------------+-------------------------------------------+---------+--------------------------+-----------------\nplugin-http-file-cost-datasource | HTTP file Cost Analysis Data Source | pyengine/plugin-http-file-cost-datasource | ENABLED | cost_analysis.DataSource | DOCKER_HUB\n```\n\n2. Register with the DataSource resource of cost-analysis.\n\n```shell\n$ spacectl exec register cost-analysis.DataSource -f register_data_source.yml\n```\n\n```yaml\n# register_data_source.yml\n---\nname: HTTP File Data Source\nservice_type: EXTERNAL\nimage: pyengine/plugin-http-file-cost-datasource\ntags: { }\ntemplate: { }\n```\n\n3. Check the registered CSV Plugin information.\n\n```shell\n$ spacectl exec get cost-analysis.DataSource -p data_source_id=\u003cdata_source_id\u003e\n\n---\ncreated_at: '2023-02-06T11:04:34.348Z'\ndata_source_id: ds-123456789012\ndata_source_type: EXTERNAL\ndomain_id: domain-123456789012\nlast_synchronized_at: '2023-02-06T16:00:08.356Z'\nname: HTTP File Data Source\nplugin_info:\n  metadata:\n    data_source_rules:\n    - actions:\n        match_service_account:\n          source: account\n          target: data.account\n      conditions: []\n      conditions_policy: ALWAYS\n      name: match_service_account\n      options:\n        stop_processing: true\n      tags: {}\n  plugin_id: plugin-http-file-cost-datasource\n  upgrade_mode: AUTO\n  version: 1.0.0.20230206.225536\nstate: ENABLED\ntags: {}\ntemplate: {}\n```\n\n4. Sets the options corresponding to the url where the csv file is located in the plugin.\n\n```shell\n$ spacectl exec update_plugin cost-analysis.DataSource -p data_source_id=\u003cdata_source_id\u003e -f update_data_source_options.yml\n```\n\n```yaml\n# update_data_source_options.yml\n---\noptions:\n  base_url:\n  - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/cost_example.csv\n  - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/custom_cost_example.csv\n  - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/example_with_billed_at.csv\n  - https://raw.githubusercontent.com/cloudforet-io/plugin-http-file-cost-datasource/master/examples/examples_of_different_headers.csv\n  field_mapper:\n    cost: TotalCost\n    currency: CurrencyCode\n    day: Day\n    month: Month\n    provider: Provider\n    usage_quantity: UsageQuantity\n    usage_type: UsageType\n    year: Year\n  default_vars:\n    currency: KRW\n  billed_at: UsageStartDate\n```\n\n5. Manually sync the cost information of the csv file in step 4.\n\n```shell\n$ spacectl exec sync cost-analysis.DataSource -p data_source_id=\u003cdata_source_id\u003e\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudforet-io%2Fplugin-http-file-cost-datasource","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudforet-io%2Fplugin-http-file-cost-datasource","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudforet-io%2Fplugin-http-file-cost-datasource/lists"}