{"id":20238404,"url":"https://github.com/pytoshka/cloud-discovery","last_synced_at":"2025-05-07T11:31:41.266Z","repository":{"id":39851530,"uuid":"329506431","full_name":"pyToshka/cloud-discovery","owner":"pyToshka","description":"Tool for discover ip addresses of nodes in cloud environments based on meta information like tags provided by the environment.","archived":true,"fork":false,"pushed_at":"2024-01-11T19:49:22.000Z","size":1690,"stargazers_count":4,"open_issues_count":4,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-18T08:49:25.279Z","etag":null,"topics":["aws","cloud-discovery","cloud-environments","gcp","kubernetes","linode"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pyToshka.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-14T04:25:44.000Z","updated_at":"2024-01-12T02:55:41.000Z","dependencies_parsed_at":"2023-02-06T15:15:48.602Z","dependency_job_id":null,"html_url":"https://github.com/pyToshka/cloud-discovery","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyToshka%2Fcloud-discovery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyToshka%2Fcloud-discovery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyToshka%2Fcloud-discovery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pyToshka%2Fcloud-discovery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pyToshka","download_url":"https://codeload.github.com/pyToshka/cloud-discovery/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252868917,"owners_count":21816936,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cloud-discovery","cloud-environments","gcp","kubernetes","linode"],"created_at":"2024-11-14T08:33:59.633Z","updated_at":"2025-05-07T11:31:39.418Z","avatar_url":"https://github.com/pyToshka.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cloud discovery\n\nTool to discover nodes in cloud environments based on meta information like tags provided by the environment.\n\n**Supported clouds**:\n\n| Provider name | Plugin name                   | Description                                                  |      |\n| ------------- | ----------------------------- | ------------------------------------------------------------ | ---- |\n| AWS           | cloud_discovery_plugin        | Retrieving informtation about EC2 instances based on tag key |      |\n| Azure         | cloud_discovery_plugin_azure  | Retrieving informtation about Azure virtual machines based on tag key |      |\n| DigitalOcean  | cloud_discovery_plugin_do     | Retrieving informtation about DigitalOcean droplets based on tag key |      |\n| GCE           | cloud_discovery_plugin_gce    | Retrieving informtation about Google cloud instances based on tag key |      |\n| Kubernetes    | cloud_discovery_plugin_k8s    | Retrieving informtation about Kubernetes pods based on tag key |      |\n| Linode        | cloud_discovery_plugin_linode | Retrieving informtation about Linodes based on tag key       |      |\n| OpenStack     | cloud_discovery_plugin_os     | Retrieving informtation about OpeStack instances based on tag key or property key |      |\n|               |                               |                                                              |      |\n|               |                               |                                                              |      |\n\nDefault cloud at this moment AWS.\n\n## Dependencies graph\n\n![cloud_discovery](img/cloud_discovery.png)\n\n## Architecture\n\nCloud discovery based on [Groundwork](https://groundwork.readthedocs.io/en/latest/).\n\nGroundwork is a Python based microframework for highly reusable applications and their components.\nIts functionality is based on exchangeable, well-documented and well-tested plugins and patterns.\n\n**Workflow**:\n\nUser run command `cloud_discovery`, application checking  input parameters and if input parameters is not presented, shows help.\n\nIf input parameter in list, checking checking option which required and output format. If output format doesn't present, application use default output format(plain) . If all parameters and options in place, application try connect to selected cloud and get information about vm/instance/pod. Application return information about requested data or exit with error\n\n## Instalation\n\nFrom GitHub\n\n```shell\npython -m pip install git+https://github.com/pyToshka/cloud-discovery.git\n```\n\nLocal installation\n\n``` shell\ngit clone git@github.com:pyToshka/cloud-discovery.git\ncd pyTerrafile\npip install .\n```\n\n## Usage\n\n### General usage:\n\n```bash\ncloud_discovery\nUsage: cloud_discovery [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n  --help  Show this message and exit.\n\nCommands:\n  aws    Find AWS instances by tag key\n  azure  Find Azure vms by tag key\n  do     Find Digitalocean droplets info by tag key\n  gce    Find GCE vms by tag key\n  k8s    Find Kubernetes pods by tag key\n  li     Find Linodes info by tag key\n  os     Find OpenStack instance info by tag key\n```\n\n#### General configuration:\n\nAlmost all providers following the same configuration strategy. As general strategy was choosed system environments for many reasons. Below providers specific usage and configuration.\n\n### Provider specific usage\n\n#### AWS\n\nThere are two way for configuration:\n\n- Use ` ~/.aws/config` file\n- System variables\n\nFor system variables make sure you exported\n\n`AWS_ACCESS_KEY_ID` - The access key for your AWS account.\n`AWS_SECRET_ACCESS_KEY` - The secret key for your AWS account.\n`AWS_DEFAULT_REGION` - The default AWS Region to use, for example, us-west-1 or us-west-2.\n\nor\n\n`AWS_PROFILE` - The default profile to use, if any. If no value is specified, Boto3 attempts to search the shared credentials file and the config file for the default profile.\n`AWS_CONFIG_FILE` - The location of the config file used by Boto3. By default this value is `~/.aws/config`. You only need to set this variable if you want to change this location.\n\n##### Example of usage:\n\n```shell\ncloud_discovery aws --help\nUsage: cloud_discovery aws [OPTIONS]\n\n  Find AWS instances by tag key\n\nOptions:\n  -t, --tag TEXT                  Tag name  [required]\n  -o, --output [json|plain|table]\n                                  Output format for received information\n  -a, --attribute [name|type|state|private_ip|public_ip]\n                                  Get single attribute\n  -r, --region TEXT               AWS region name. Default us-east-1\n  --help                          Show this message and exit.\n```\n\nBelow example how to get all instances with tag name `project`\n\n```bash\ncloud_discovery aws  -t project\n```\n\nBy default output is `plain`\n\n```shell\nname: test-vm\ntype: t3.large\nstate: running\nprivate_ip: 127.0.0.1\npublic_ip: 8.8.8.8\n------\n\n```\n\n**Json**\n\nCommand  `cloud_discovery aws -t project -o json `\n\nOutput:\n\n```json\n[\n    {\n        \"name\": \"test-vm\",\n        \"private_ip\": \"127.0.0.1\",\n        \"public_ip\": \"8.8.8.8\",\n        \"state\": \"running\",\n        \"type\": \"t3.large\"\n    }\n]\n```\n\n**Table**\n\nCommand `cloud_discovery aws -t project -o table`\n\nOutput:\n\n```shell\nname                        | type     | state   | private_ip   | public_ip\n--------------------------- | -------- | ------- | ------------ | --------------\ntest-vm                     | t3.large | running | 127.0.0.1    | 8.8.8.8\n```\n\n\n\nAs one of future `cloud_discovery` can return elements as string, at this moment tool supports only\n\n- name\n- type\n- state\n- private_ip\n- public_ip\n\n##### Example:\n\n```shell\ncloud_discovery aws  -t project -o table -a private_ip\n127.0.0.1\n```\n\n#### Azure\n\n##### System environments:\n\n`AZURE_SUBSCRIPTION_ID` - The Azure subscription id\n`AZURE_CLIENT_ID` - the service principal's client ID\n`AZURE_CLIENT_SECRET` - one of the service principal's client secrets\n`AZURE_TENANT_ID` - ID of the service principal's tenant. Also called its 'directory' ID.\n\n**Example**:\n\n```shell\nexport AZURE_SUBSCRIPTION_ID=\"aa11bb33-cc77-dd88-ee99-0918273645aa\"\nexport AZURE_TENANT_ID=\"00112233-7777-8888-9999-aabbccddeeff\"\nexport AZURE_CLIENT_ID=\"12345678-1111-2222-3333-1234567890ab\"\nexport AZURE_CLIENT_SECRET=\"abcdef00-4444-5555-6666-1234567890ab\"\n```\n\n##### Example of usage:\n\n```bash\ncloud_discovery azure --help\nUsage: cloud_discovery azure [OPTIONS]\n\n  Find Azure vms by tag key\n\nOptions:\n  -t, --tag TEXT                  Tag name  [required]\n  -o, --output [json|plain|table]\n                                  Output format for received information\n  -a, --attribute [name|type|state|private_ip|public_ip]\n                                  Get single attribute\n  -r, --resource_group TEXT       Azure resource group name\n  --help                          Show this message and exit.\n```\n\nOutput format the same as for `AWS`\n\n#### Google cloud engine\n\n##### Configuration:\n\nBefor starting make sure `Cloud SDK` installed and configured.\n\n How to install and configure [Cloud SDK](https://cloud.google.com/sdk/docs/quickstart)\n\nSetup system environment `GOOGLE_APPLICATION_CREDENTIALS` with Your [authentication credentials](https://cloud.google.com/docs/authentication/getting-started)\n\n##### Example:\n\n```shell\n\texport GOOGLE_APPLICATION_CREDENTIALS=\"/tmp/google.json\"\n```\n\n##### Example of usage:\n\n```shell\ncloud_discovery gce --help\nUsage: cloud_discovery gce [OPTIONS]\n\n  Find GCE vms by tag key\n\nOptions:\n  -t, --tag TEXT                  Tag name  [required]\n  -o, --output [json|plain|table]\n                                  Output format for received information\n  -a, --attribute [name|type|state|private_ip|public_ip]\n                                  Get single attribute\n  -r, --region TEXT               Zone name for example us-central1-a\n                                  [required]\n\n  -p, --project TEXT              Project name  [required]\n  --help                          Show this message and exit.\n```\n\nOutput format the same as for `AWS`\n\n#### DigitalOcean\n\n##### Configuration:\n\nMake sure you exported `DIGITALOCEAN_ACCESS_TOKEN` as system variable\n\n##### Example:\n\n```shell\nexport DIGITALOCEAN_ACCESS_TOKEN=\"dddd-dd-dddd\"\n```\n\n##### Example of usage:\n\n```shell\ncloud_discovery do --help\nUsage: cloud_discovery do [OPTIONS]\n\n  Find Digitalocean droplets info by tag key\n\nOptions:\n  -t, --tag TEXT                  Tag name  [required]\n  -o, --output [json|plain|table]\n                                  Output format for received information\n  -a, --attribute [name|type|state|private_ip|public_ip|ipv6_address]\n                                  Get single attribute\n```\n\nOutput the same as for  `AWS`\n\n#### Kubernetes\n\n##### Configuration:\n\nKubernetes provider use dynamic kubernetes client library. Before start you should be connected to Kubernetes cluster via kubectl, `cloud_discovery` will use current cluster and context\n\n##### Example of usage:\n\n```shell\ncloud_discovery k8s --help\nUsage: cloud_discovery k8s [OPTIONS]\n\n  Find Kubernetes pods by tag key\n\nOptions:\n  -l, --label TEXT                Pods label  [required]\n  -o, --output [json|plain|table]\n                                  Output format for received information\n  -a, --attribute [name|namespace|state|private_ip|public_ip]\n                                  Get single attribute\n  -t, --top                       Show pods statistic\n  -s, --sort [cpu|memory]         Sort by value\n  -m, --limit INTEGER             Number of pods in top table\n  --help                          Show this message and exit.\n```\n\nOutput fields\n\n- `name` - Kubernetes pod name\n- `namespace` - Kubernetes namespace name\n- `state` - Pod current status\n- `private_ip` - Pod private ip\n- `public_ip` - Not public ip really. Kubernetes ingress hostname\n\nTop like output per pod. You can get/sort and limit statictic about memory and cpu usage in realtime per pod.\n\nExample of running\n\n``` shell\ncloud_discovery k8s -l k8s-app=kube-proxy -t -s memory --limit 3\n\n```\n\nIt means getting memory usage for application with label `k8s-app=kube-proxy` sorted by memory usage and show me top 3 pod\n\n#####  Output:\n\n![k8s_top](img/k8s_top.png)\n\n#### Linode\n\n##### Configuration:\n\nMake sure you exported LINODE_ACCESS_TOKEN as system variable\n\n##### Example of usage:\n\n```shell\ncloud_discovery li --help\nUsage: cloud_discovery li [OPTIONS]\n\n  Find Linodes info by tag key\n\nOptions:\n  -t, --tag TEXT                  Tag name  [required]\n  -o, --output [json|plain|table]\n                                  Output format for received information\n  -a, --attribute [name|type|state|private_ip|public_ip|ipv6_address]\n                                  Get single attribute\n  --help                          Show this message and exit.\n\n```\n\nOutput the same as for  `AWS`\n\n#### OpenStack\n\n##### Configuration:\n\nMake sure you have next system environments:\n\n`OS_AUTH_URL` - The Identity authentication URL.\n\n`OS_USERNAME` - The Username to login with\n\n`OS_PASSWORD` - The Password to login with\n\n`OS_PROJECT_ID` - The ID of the Tenant (Identity v2) or Project (Identity v3) to login with\n\n`OS_USER_DOMAIN_NAME` - The domain name where the user is located.\n\n`OS_INTERFACE` -OpenStack interface type\n\n`OS_REGION_NAME` - The region of the OpenStack cloud to use\n\nAll variables could be exported from OpenStack [rc file](https://docs.openstack.org/mitaka/install-guide-obs/keystone-openrc.html)\n\n##### Example of usage:\n\n```shell\ncloud_discovery os --help\nUsage: cloud_discovery os [OPTIONS]\n\n  Find OpenStack instance info by tag key\n\nOptions:\n  -t, --tag TEXT                  Tag or property name  [required]\n  -o, --output [json|plain|table]\n                                  Output format for received information\n  -a, --attribute [name|type|state|private_ip|public_ip]\n                                  Output format for received information\n  --help                          Show this message and exit.\n```\n\n`cloud_discovery`  try to get information based on tag or properties\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpytoshka%2Fcloud-discovery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpytoshka%2Fcloud-discovery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpytoshka%2Fcloud-discovery/lists"}