{"id":42841141,"url":"https://github.com/kosmas-valianos/gcloudfilter","last_synced_at":"2026-01-30T11:50:50.469Z","repository":{"id":176736633,"uuid":"654060526","full_name":"kosmas-valianos/gcloudfilter","owner":"kosmas-valianos","description":"Filter GCP projects locally instead of doing expensive API calls","archived":false,"fork":false,"pushed_at":"2024-08-30T08:48:48.000Z","size":104,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-01T00:25:51.477Z","etag":null,"topics":["cloud-resource-manager","gcp","go","golang","google-cloud"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kosmas-valianos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-15T09:57:15.000Z","updated_at":"2024-08-30T08:39:37.000Z","dependencies_parsed_at":"2023-12-16T12:50:31.742Z","dependency_job_id":"b13a719f-62e0-4f60-bd9a-dea2d7d7a567","html_url":"https://github.com/kosmas-valianos/gcloudfilter","commit_stats":{"total_commits":44,"total_committers":1,"mean_commits":44.0,"dds":0.0,"last_synced_commit":"e939773ba357eb9ffeb8399b48ac428e099fe580"},"previous_names":["kosmas-valianos/gcloudfilter"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/kosmas-valianos/gcloudfilter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kosmas-valianos%2Fgcloudfilter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kosmas-valianos%2Fgcloudfilter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kosmas-valianos%2Fgcloudfilter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kosmas-valianos%2Fgcloudfilter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kosmas-valianos","download_url":"https://codeload.github.com/kosmas-valianos/gcloudfilter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kosmas-valianos%2Fgcloudfilter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28912077,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-30T08:15:08.179Z","status":"ssl_error","status_checked_at":"2026-01-30T08:14:31.507Z","response_time":66,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-resource-manager","gcp","go","golang","google-cloud"],"created_at":"2026-01-30T11:50:49.871Z","updated_at":"2026-01-30T11:50:50.464Z","avatar_url":"https://github.com/kosmas-valianos.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gcloudfilter\nDefine a lexer and parser to enable filtering of GCP projects, instances and forwarding rules **locally** instead of doing expensive API calls. Especially the API for the projects has a **low quota** therefore it is very easy to end up getting rate limited in case your application has to perform many queries. A typical application would specify the filter in the `Query`/`Filter` field of the `Request` object parameter and do an API call to retrieve the resources that match that `Query`/`Filter`. Instead of spamming API calls with the imminent danger of getting rate limited you can now request **all** the resources you want at **every X interval** and use the `FilterProjects()`/`FilterInstances()`/`FilterForwardingRules` from this package to filter locally by running the query on the cached resources. In that way the API calls are drastically reduced to a constant 1 per interval instead of 1 per query request! For example an application that has to make 10000 requests it would have to make 10000 API calls but now it will be only 1... The grammar and syntax are specified in [gcloud topic filters](https://cloud.google.com/sdk/gcloud/reference/topic/filters)\n\n## Installation\n```\ngo get github.com/kosmas-valianos/gcloudfilter\n```\n\n## Usage/Example\nA lot of raw queries can be seen in the unit tests.\n\nThe following application downloads and caches all the projects using `SearchProjects()` with 60 seconds update interval. The user can run endless projects' queries using the standard input without worrying about any quota limits as the filtering is happening locally using the `FilterProjects()` on the cached projects.\n\n```golang\ntype projectsGCP struct {\n\tresources []*resourcemanagerpb.Project\n\terr       error\n\tRWMutex   *sync.RWMutex\n}\n\nvar projects projectsGCP = projectsGCP{RWMutex: \u0026sync.RWMutex{}}\n\nfunc updateProjects() {\n\tprojects.RWMutex.Lock()\n\tprojects.err = nil\n\tdefer func() {\n\t\tif projects.err != nil {\n\t\t\tfmt.Println(projects.err)\n\t\t}\n\t\tprojects.RWMutex.Unlock()\n\t}()\n\tprojects.resources = make([]*resourcemanagerpb.Project, 0)\n\n\tctx := context.Background()\n\tprojectsClient, err := resourcemanager.NewProjectsClient(ctx)\n\tif err != nil {\n\t\tprojects.err = fmt.Errorf(\"projects update: failed to create client: %w\", err)\n\t\treturn\n\t}\n\tdefer projectsClient.Close()\n\n\tvar req resourcemanagerpb.SearchProjectsRequest\n\tit := projectsClient.SearchProjects(ctx, \u0026req)\n\tvar sb strings.Builder\n\tsb.Grow(256)\n\tsb.WriteString(\"Projects cached: \")\n\tfor {\n\t\tresp, err := it.Next()\n\t\tif err == iterator.Done {\n\t\t\tbreak\n\t\t}\n\t\tif err != nil {\n\t\t\tprojects.err = fmt.Errorf(\"projects update: failed to advance iterator: %w\", err)\n\t\t\tbreak\n\t\t}\n\t\tprojects.resources = append(projects.resources, resp)\n\t\tsb.WriteString(fmt.Sprintf(\"%v \", resp.ProjectId))\n\t}\n\tfmt.Println(sb.String())\n}\n\nfunc updateProjectsTicker() {\n\tticker := time.NewTicker(60 * time.Second)\n\tdefer ticker.Stop()\n\tfor t := range ticker.C {\n\t\tfmt.Printf(\"GET all the projects to cache them at %v\\n\", t.Format(time.DateTime))\n\t\tupdateProjects()\n\t}\n\n}\n\nfunc readInput() {\n\tfor {\n\t\treader := bufio.NewReader(os.Stdin)\n\t\tfmt.Println(\"Enter GCP project filter:\")\n\t\tstr, err := reader.ReadString('\\n')\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t\tstr = str[:len(str)-1]\n\t\tfmt.Printf(\"Filter query: %v\\n\", str)\n\t\tprojects.RWMutex.RLock()\n\t\tprojectsFiltered, err := gcloudfilter.FilterProjects(projects.resources, str)\n\t\tprojects.RWMutex.RUnlock()\n\t\tif err != nil {\n\t\t\tlog.Fatal(fmt.Errorf(\"applying project filter: %v failed: %w\", str, err))\n\t\t}\n\t\tvar sb strings.Builder\n\t\tsb.Grow(256)\n\t\tsb.WriteString(\"Projects after filtering: \")\n\t\tfor _, project := range projectsFiltered {\n\t\t\tsb.WriteString(fmt.Sprintf(\"%v \", project.ProjectId))\n\t\t}\n\t\tfmt.Println(sb.String())\n\t}\n}\n\nfunc main() {\n\tupdateProjects()\n\tgo updateProjectsTicker()\n\treadInput()\n}\n```\n\n```\n# ./gcloudfilter_example \nProjects cached: corelogic-30-603 appgate-test appgate-dev product-team-222016 \nEnter GCP project filter:\nid=(\"appgate-dev\" \"foo\") AND (-labels.boo:* OR labels.envy:*)\nFilter query: id=(\"appgate-dev\" \"foo\") AND (-labels.boo:* OR labels.envy:*)\nProjects after filtering: appgate-dev \nEnter GCP project filter:\n\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkosmas-valianos%2Fgcloudfilter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkosmas-valianos%2Fgcloudfilter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkosmas-valianos%2Fgcloudfilter/lists"}