{"id":50714637,"url":"https://github.com/nullstone-modules/gcp-vertexai-invoker","last_synced_at":"2026-06-09T18:01:33.206Z","repository":{"id":363587255,"uuid":"1263309945","full_name":"nullstone-modules/gcp-vertexai-invoker","owner":"nullstone-modules","description":"Enables a GCP app to invoke models in GCP Vertex AI","archived":false,"fork":false,"pushed_at":"2026-06-09T13:27:33.000Z","size":7,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-06-09T15:20:55.961Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nullstone-modules.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-08T20:39:58.000Z","updated_at":"2026-06-09T13:38:04.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/nullstone-modules/gcp-vertexai-invoker","commit_stats":null,"previous_names":["nullstone-modules/gcp-vertexai-invoker"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/nullstone-modules/gcp-vertexai-invoker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullstone-modules%2Fgcp-vertexai-invoker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullstone-modules%2Fgcp-vertexai-invoker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullstone-modules%2Fgcp-vertexai-invoker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullstone-modules%2Fgcp-vertexai-invoker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nullstone-modules","download_url":"https://codeload.github.com/nullstone-modules/gcp-vertexai-invoker/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nullstone-modules%2Fgcp-vertexai-invoker/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34118757,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-09T02:00:06.510Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-09T18:01:31.895Z","updated_at":"2026-06-09T18:01:33.195Z","avatar_url":"https://github.com/nullstone-modules.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GCP Vertex AI Invoker\n\nA least-privilege Nullstone capability that grants a workload's runtime identity exactly enough IAM to **call** Vertex AI models — and nothing else.\n\n## Overview\n\nThe common path is to hand a workload `roles/aiplatform.user` so it can reach Vertex AI. That role also grants the ability to launch training jobs, deploy endpoints, and run batch prediction — capabilities an inference client never exercises, but that turn a compromised credential into a cost-bomb and an arbitrary-compute foothold.\n\nThis capability grants the single permission an inference workload actually uses (a LiteLLM proxy, an app calling Gemini / Claude / Kimi, an embedding service):\n\n- `aiplatform.endpoints.predict` — model invocation. Covers `generateContent` / `streamGenerateContent` (Gemini), `rawPredict` / `streamRawPredict` (Claude, Kimi, and other partner/MaaS models), embeddings, and calls to self-deployed endpoints. It also authorizes the `countTokens` and `computeTokens` methods, so token-based usage accounting works without any additional grant. (`computeTokens`/`countTokens` are API methods, not IAM permissions — there is no separate `aiplatform.endpoints.computeTokens` permission to grant.)\n\nIt packages this into a project-scoped custom IAM role and binds that role to the target workload identity. The worst-case abuse of the resulting credential is bounded inference spend — which you cap further with Vertex quotas and application-layer budgets.\n\n## What it deliberately does NOT grant\n\n- Custom / hyperparameter-tuning jobs (`aiplatform.customJobs.*`)\n- Endpoint creation or model deployment (`aiplatform.endpoints.create`, `aiplatform.models.upload`, `…deployModel`)\n- Batch prediction jobs (`aiplatform.batchPredictionJobs.*`)\n- Dataset, featurestore, and pipeline management\n- Context-cache resource management (`aiplatform.cachedContents.*`)\n- Any IAM or project mutation\n\nThis exclusion list is the product. The capability is a narrow edge between a workload and Vertex inference, not a general grant of \"use Vertex.\"\n\n## How it works\n\n- Creates a project-level custom role (`vertexAiInvoker` by default) containing the single permission above.\n- Binds that role to the workload's runtime service account:\n  - **As a capability attached to an app block**, it auto-discovers and binds the app's runtime service account (e.g., your LiteLLM service running on GKE via Workload Identity).\n  - **Standalone**, you pass the target service account email explicitly.\n\nIAM here is project-scoped and **not** region-specific. Which Vertex region/location a request targets is configured at the application layer (e.g., LiteLLM's `vertex_location`), not in this capability. Granting predict at the project level keeps the role stable across region changes.\n\n## Prerequisites\n\n- Vertex AI API (`aiplatform.googleapis.com`) enabled on the project.\n- **Partner/MaaS models entitled in Model Garden.** Invoking Claude, Kimi, or other third-party models requires a one-time entitlement (accept commercial terms on the model card), which needs `roles/consumerprocurement.entitlementManager`. Entitlement is per-project and per-region.\n\n\u003e **This capability intentionally does not perform model entitlement.** Entitlement enables new paid third-party models and is a privileged, human-in-the-loop action. Keeping it out of an automated runtime grant is deliberate: the running workload should be able to *call* already-entitled models, never to *enable* new ones. Handle entitlement once through a separate admin path with the entitlement-manager role.\n\n## Usage\n\nAttach as a capability to the app that invokes Vertex (e.g., your LiteLLM proxy). When attached, the target identity is resolved automatically from the app block — no service account wiring required.\n\nFor standalone use, supply `project_id` and `target_service_account`.\n\n## Inputs\n\nThere are no inputs.\n\n## Outputs\n\nThere are no outputs.\n\n## Security notes\n\nThe runtime credential is ambient — Workload Identity keeps a valid token in the pod continuously — and the workload it backs (an API gateway / model proxy) is typically the most exposed surface in the stack. Under `roles/aiplatform.user`, a stolen token can provision GPU/TPU capacity and run arbitrary containers via custom jobs. Under this role, it can only invoke models.\n\nThe tight scope also makes detection cheap: any non-predict Vertex action originating from this identity is an unambiguous anomaly, rather than something that blends into a broad role's legitimate traffic.\n\nPair with **Vertex quotas** (cap request volume) and **application-layer budgets** such as LiteLLM virtual-key spend limits (cap cost). IAM removes the dangerous categories of action; quotas and budgets bound the one category that remains.\n\nFor SOC2, this maps directly to least-privilege / access-control evidence: the custom role definition is the auditable artifact, and its permission set is minimal and explicit.\n\n## Extending\n\nIf you enable additional Vertex features, grant additive permissions rather than widening back to `aiplatform.user`:\n\n- **Batch prediction** → `aiplatform.batchPredictionJobs.*`\n- **Gemini explicit context caching** (managing `CachedContent`) → `aiplatform.cachedContents.*`. Note: Anthropic prompt caching is an inline `cache_control` block inside the request payload and needs no extra permission.\n- **Callback logging** to GCS / BigQuery → the relevant storage / BigQuery permissions, ideally scoped to a separate identity rather than folded in here.\n\nEach missing permission surfaces as a precise `PERMISSION_DENIED` naming the exact permission, so you can grant only what's required.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnullstone-modules%2Fgcp-vertexai-invoker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnullstone-modules%2Fgcp-vertexai-invoker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnullstone-modules%2Fgcp-vertexai-invoker/lists"}