{"id":34527665,"url":"https://github.com/eclipse-xfsc/custom-policy-agent","last_synced_at":"2026-05-28T00:31:23.183Z","repository":{"id":289361069,"uuid":"963958444","full_name":"eclipse-xfsc/custom-policy-agent","owner":"eclipse-xfsc","description":"The policy service provides REST API to evaluate/execute OPA policies written in the Rego language.","archived":false,"fork":false,"pushed_at":"2025-06-04T19:42:22.000Z","size":17417,"stargazers_count":0,"open_issues_count":5,"forks_count":0,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-06-05T00:04:04.144Z","etag":null,"topics":["goa","golang","opa","pdp","rego","tsa"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eclipse-xfsc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-10T13:25:29.000Z","updated_at":"2025-06-04T19:42:24.000Z","dependencies_parsed_at":"2025-06-04T20:50:31.421Z","dependency_job_id":"1697a7b4-12f7-4bca-991f-11de30fe3f43","html_url":"https://github.com/eclipse-xfsc/custom-policy-agent","commit_stats":null,"previous_names":["eclipse-xfsc/custom-policy-agent"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/eclipse-xfsc/custom-policy-agent","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eclipse-xfsc%2Fcustom-policy-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eclipse-xfsc%2Fcustom-policy-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eclipse-xfsc%2Fcustom-policy-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eclipse-xfsc%2Fcustom-policy-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eclipse-xfsc","download_url":"https://codeload.github.com/eclipse-xfsc/custom-policy-agent/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eclipse-xfsc%2Fcustom-policy-agent/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33589683,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-27T02:00:06.184Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["goa","golang","opa","pdp","rego","tsa"],"created_at":"2025-12-24T05:24:16.496Z","updated_at":"2026-05-28T00:31:23.177Z","avatar_url":"https://github.com/eclipse-xfsc.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![pipeline status](https://github.com/eclipse-xfsc/custom-policy-agent/badges/main/pipeline.svg)](https://github.com/eclipse-xfsc/custom-policy-agent/-/commits/main)\n[![coverage report](https://github.com/eclipse-xfsc/custom-policy-agent/badges/main/coverage.svg)](https://github.com/eclipse-xfsc/custom-policy-agent/-/commits/main)\n\n# Policy Service\n\nThe policy service provides REST API to evaluate/execute \n[OPA](https://www.openpolicyagent.org/) policies written in\nthe Rego language. The policy engine is extended with custom \nfunctions available for use through the Rego runtime during\npolicy execution. The service also provides endpoints for exporting\nand importing policy bundles, for subscribing for policy changes and endpoints for policy\nadministration (e.g. listing policies, lock/unlock specific policy).\n\nIt is developed using the [Goa v3](https://goa.design/) framework\nand uses the [Go OPA framework](https://github.com/open-policy-agent/opa) \nas a library.\n\n\u003e A helper script named `goagen.sh` can be found inside the root directory of\n\u003e the service. It can be used to generate the transport layer code from the\n\u003e Goa DSL definitions in the [design](./design) directory. The script should\n\u003e be executed everytime the design definitions are updated. It also generates\n\u003e updated OpenAPI documentation from the DSL.\n\n[OpenAPI documentation](https://github.com/eclipse-xfsc/custom-policy-agent/-/blob/main/gen/http/openapi3.json)\n\nIn the local docker-compose environment, the Swagger URL is available at http://localhost:8081/swagger-ui/ \n\n### High-level Overview\n\n```mermaid\nflowchart LR\n\tA([client]) -- HTTP --\u003e B[Policy API]\n\tsubgraph policy\n\t\tB --\u003e C[(policies DB)]\n\tend\n\tC --sync--- D[Git Server]\n```\n\n### Configuration\n\nThe Policy service is configured using the [Configuration File](./internal/config/config.go).\nAll configurations are expected as Environment variables specified in the\nconfiguration file. For managing the configuration data from ENV variables,\n[envconfig library](https://github.com/kelseyhightower/envconfig) is used.\n\n\n### Policy Evaluation\n\nThe policy service exposes HTTP endpoints to evaluate/execute policies.\nThe endpoint interface is conformant to the TSA requirements document.\n\nTo evaluate a policy a GET or POST request is sent to the evaluation URL.\nThe example URL below is given for the local docker-compose environment. \nThe `host` and `port` parts will be different for the different environments.\n\n```\n# URL with example policy repository, group, name and version\nhttp://localhost:8081/policy/policies/xfsc/didresolve/1.0/evaluation\n\n# URL with parameter placeholders\nhttp://localhost:8081/policy/{repository}/{group}/{policy}/{version}/evaluation\n```\n\nThere are four parameters in the URL specifying which exact policy \nshould be evaluated - `repository`, `group`, `policy` and `version`. These parameters \nare also important during policy development (see below) as `group` \nand `policy` **must** be used as package name inside the policy \nsource code file.\n\nThe body of the POST request can be empty, but if it's not empty, it \n**must** be JSON. It is passed directly to the policy execution runtime. \nInside the policy it is accessed with the global variable name `input`. \nFor example, if you pass to the evaluation endpoint the following JSON, \nit will be accessible by `input.message`:\n```json\n{\n  \"message\": \"hello world\"\n}\n```\n\nHere is a complete example CURL request:\n```shell\ncurl -X POST http://localhost:8081/policy/policies/xfsc/didresolve/1.0/evaluation -d '{\"message\":\"hello world\"}'\n```\n\nNote: If the version or any other subpath is missing in the folder structure (e.g. in combination with the repo feature), the service appends a \"temp\" group, which results in another structure of the URL. In this case the service fires \"result empty\" responses, because the package name must be corrected as well to the temp path. \n\n### Policy output JSON schema validation\n\nThe policy service exposes HTTP endpoint to validate the output of the policy. It uses\n[JSON Schema](https://json-schema.org/) to validate the JSON schema of the output\nof the policy. To execute the validation procedure, a new HTTP URL is automatically generated.\n\n```\nEvaluation URL pattern: {service_addr}/{repo}/{group}/{policyname}/{version}/evaluation\nValidation URL pattern: {service_addr}/{repo}/{group}/{policyname}/{version}/validation\n```\n\nIn order to use the validation endpoint, `output-schema.json` file following the [JSON Schema](https://json-schema.org/)\nspecification must be present in the policy repository directory. For more information on policy development refer to:\n[Policy Development](#Policy-Development)\n\n### Policy Locking\n\nThe service exposes HTTP endpoints to lock and unlock policies. Locking a policy\nmeans that it's not allowed for evaluation (execution). Unlocking a policy allows\nits evaluation/execution to proceed normally.\n\nLock a policy with POST request:\n```shell\ncurl -X POST http://localhost:8081/policy/policies/xfsc/didresolve/1.0/lock\n```\n\nUnlock a policy with DELETE request:\n```shell\ncurl -X DELETE http://localhost:8081/policy/policies/xfsc/didresolve/1.0/lock\n```\n\n### Policy Bundles\n\nA policy bundle contains a Policy source code, static data, configuration and some\nadditional meta-info necessary for verifying the origin of the policy (who signed the bundle).\n\nThe service allows to export a policy bundle by making HTTP request against a policy URL\nof the same format as is used for evaluation, but instead of using the `evaluation` word\nat the end, you replace it with `export`. That will pack the policy as ZIP bundle, \nand the ZIP bundle will be returned to the client.\n\n```shell\nwget http://localhost:8081/repository/policy/xfsc/didresolve/1.0/export\n```\n\n\u003eSee [here](./doc/policy_bundles.md) for more detailed overview of \n\u003epolicy bundles export/import.\n\n### Policy Storage\n\nPolicies (rego source code and metadata) are stored in a storage, which is an interface\nand different implementations could be used. You can check the interface\n[here](./internal/service/policy/storage.go).\n\nCurrently, there are two implementations of the storage interface:\n - [MongoDB](./doc/mongodb_storage.md)\n - [Memory](./doc/memory-storage.md)\n\nMatrix for storage feature availability:\n\n **Feature** | **MongoDB** | **Memory**\n--- |--------|------------\nPolicy Lock/Unlock | Yes    | Yes*\nChange Notifications | Yes    | Yes*\nStorage extension functions | Yes | Yes*\nAutomatic synchronization | Yes | N/A\nBundle import/export | Yes | Yes*\n\n\u003e `*` Functionality is available only for the current instance of the policy service. Synchronization between\n\u003e instances of the policy service is not available.\n\nIn order to use another storage technology, one should implement the [Storage interface](./internal/service/policy/storage.go).\n\n```mermaid\nflowchart LR\n\tA[Policy\\nDeveloper] --git push/merge--\u003e B[Git branch]\n\tsubgraph Git Server\n\t\tB --\u003e C[example/1.0/policy.rego]\n\t\tB --\u003e D[example/2.0/policy.rego]\n\t\tB --\u003e G[example/3.0/policy.rego]\n\tend\n\tC --\u003e E[Sync]\n\tD --\u003e E[Sync]\n\tG --\u003e E[Sync]\n\tE --update--\u003e F\n\tsubgraph policy service\n\t\tF[(policies DB)]\n\tend\n```\n\n### Subscribe for Policy Changes\n\nThe service allows external clients to subscribe for policy changes.\nA change event happens when policy source code, static data, export configuration or\nany state-related data is updated. Two subscription mechanisms are supported.\n\n##### Cloud Events  \n\nWhen policy change event happens, a cloud event is sent using the CloudEvents Go SDK.\nIt uses standard format for encoding/decoding events and typically uses a message\nqueue under the hood for delivery of events. Our implementation is using NATS, but\nany broker supporting the CloudEvents specification should work. \n\n##### Web Hooks\n\nThis mechanism allow clients to express interest in change events for a specific policy\nand when such change event is triggered, the policy service shall notify the client.\nA client subscribes for policy change by making an HTTP request to a policy endpoint\nwith the suffix `notifychage`, for example:\n```\n$ curl https://mysvc.com/policy/policies/example/policyname/1.0/notifychange \n-d '{\"webhook_url\":\"https://url-to-call-when-policy-changes.com\"}'\n```\n\nThis request creates a record in storage, so the Policy service will know which \nwebhook URL to call when a policy change event happens. \n\nThe event type that will be received by the subscriber is per policy and is sent\nas JSON structure like:\n```json\n{\n  \"repository\": \"policies\",\n  \"group\": \"example\",\n  \"name\" : \"mypolicy\",\n  \"version\": \"1.0\"\n}\n```\nIt is defined in [notify.go](./internal/notify/notify.go)\n\n### Policy Admin API\n\nThe API allows to inspect the internal state of the policies without requiring\ndirect access to the policy database. This could be useful for debugging or checking\nwhat is the current source code or static data for a given policy. The endpoint is called\nwith HTTP GET requests and supports basic filtering and expansion via query params:\n```\nGET /v1/policies?policyName=hello\u0026locked=true\u0026rego=true\u0026data=true\u0026dataConfig=true\n```\n\n\u003e All query parameters are optional.\n\n### Policy Development\n\n* [Policy Extensions Functions](./doc/policy_development.md)\n\nPolicies are written in the [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) \nlanguage. Please refer to the [OPA documentation](https://www.openpolicyagent.org/docs/latest/)\nfor detailed overview of Rego and OPA capabilities.\n\n**Some conventions *must* be followed when writing policies.**\n\n1. The filename of the policy *must* follow rules for the naming and directory structure:\nthe `group`, `policy name` and `version` are directories inside the Git repo and policy file *must* be named\n`policy.rego`.  For example: `/xfsc/example/1.0/policy.rego`.\n2. In the same directory there could be a data file containing static JSON, which is automatically \navailable for use during policy evaluation by using the `data` variable. The file *must* be named `data.json`. \nExample: `/xfsc/example/1.0/data.json`\n3. In the same directory there could be a configuration file containing information for getting static JSON\ndata from external URL. The file *must* be named `data-config.json`.\nExample: `/xfsc/example/1.0/data-config.json`\n\u003e Note that there should only be one of the two files `data.json` or `data-config.json` in the same directory.\n\u003e If both files exist in the same directory tha data from the `data.json` file will be eventually overwritten by the data\n\u003e acquired using the configuration from the `data-config.json` file.\n4. In the same directory there could be a configuration file containing JSON schema for validating the policy\nevaluation output. The file *must* be named `output-schema.json`.\nExample: `/xfsc/example/1.0/output-schema.json`\n5. The policy package name inside the policy source code file *must* exactly match\nthe `group` and `policy` (name) of the policy.\n\n##### *What does it mean?*\n\n- Let's see an example for the 1st convention.\n```\npackage xfsc.example\n\nallow {\n    input.message == \"hello world\"\n}\n```\n\nNext, the filename must be `/xfsc/example/1.0/policy.rego`. When such file is synchronized\nwith the policy service (storage), the naming convention allows the service to understand\nwhich part is the policy group, which part is policy name and which part is version.\n\nIf we create the above policy and store it in the Git repo as `/xfsc/example/1.0/policy.rego`,\nafter the Git server is synchronized with the Policy Storage, the policy service will\nautomatically expose URLs for working with the policy at:\n```\nhttp://localhost:8081/policy/xfsc/example/1.0/evaluation\nhttp://localhost:8081/policy/xfsc/example/1.0/lock\n```\n- The 2nd rule for static data file naming is to make sure that file `/xfsc/example/1.0/data.json`\nis passed and is available to the evaluation runtime when a policy is evaluated at URL:\n```\nhttp://localhost:8081/policy/xfsc/example/1.0/evaluation\n```\nStatic data is accessed within the Rego policy with `data.someKey`.\nExample: If the `/xfsc/example/1.0/data.json` file is:\n```json\n{\n  \"name\": \"some name\"\n}\n```\none could access the data using `data.name` within the Rego source code.\n\n- The 3rd rule for configuration file is to provide configurations for getting static JSON data from external URL.\nThe file must contain a URL, an HTTP method and a period, after which an HTTP request is made to get the latest data.\n\u003e The period must be added as duration e.g. `10h`, `1h30m` etc.\n\nThe file MAY contain body for the request.\nExample file contents:\n```json\n{\n  \"url\": \"http://example.com/data.json?page=3\",\n  \"method\": \"GET\",\n  \"period\": \"10h\",\n  \"body\": {\n    \"key\": \"value\"\n  }\n}\n```\nThis means that every 10 hours an HTTP request is going to be made on the given URL, with `GET` method and the result is going\nto be stored as static data for this policy and passed during evaluation.\n\n- The 4th rule for policy output schema validation is to provide a JSON schema which will be\nused to validate the output of the policy.\n\nExample file contents:\n```json\n{\n  \"type\": \"object\",\n  \"properties\": {\n    \"foo\": {\n      \"type\": \"string\",\n      \"minLength\": 5\n    }\n  }\n}\n```\n\nThis policy output would be valid: `{\"foo\":\"barbaz\"}`.\nThis policy output would be invalid: `{\"foo\":\"bar\"}`.\n\n- The 5th rule for package naming is needed so that a generic evaluation function\ncan be mapped and used for evaluating all kinds of different policies. Without a \npackage naming rule, there's no way the service can automatically generate HTTP \nendpoints for working with arbitrary dynamically uploaded policies.\n\n### Access HTTP Headers inside a policy\n\nHTTP request headers are passed to the evaluation runtime on each request. They can be\naccessed through a built-in extension function named `external.http.header()`. It accepts as argument\nthe name of the header in [Canonical](https://golangbyexample.com/canonical-http-header-key/) \nformat. For example, inside Rego the value of a header named `Authorization` can be retrieved\nas follows:\n```\npackage example.example\n\nauth := external.http.header(\"Authorization\")\n```\n\n\u003eHeader names are passed to the Rego runtime in Canonical format. This means that the \n\u003efirst character and any characters following a hyphen are uppercase and the rest \n\u003eare lowercase.\n\nMore examples, if the policy service receive a request with the following headers:\n```\naccept-encoding: gzip, deflate\nAccept-Language: en-us\nfOO: Bar\nx-loCATion: Baz\n```\nInside a policy these headers could be accessed as follows:\n```\naccept_encoding := external.http.header(\"Accept-Encoding\")\naccept_language := external.http.header(\"Accept-Language\")\nfoo := external.http.header(\"Foo\")\nlocation := external.http.header(\"X-Location\")\n```\n\n### Policy Extensions Functions\n\nA brief documentation for the available Rego extensions functions\nwhich can be used during policy development.\n\n[Policy Extensions Functions](./doc/policy_development.md)\n\nYou can also look at the source code in package [`regofunc`](./internal/regofunc) to understand the\ninner-working and capabilities of the extension functions.\n\n### Build\n\n##### Local binary\nTo make the service binary locally, you can run the following command from the root\ndirectory (you must have [Go](https://go.dev/) installed):\n```shell\ngo build -o policy ./cmd/policy/...\n```\n\n##### Docker image\n\nYou can see the Dockerfile of the service under the [deployment](./deployment) directory.\nThere is one Dockerfile for use during local development with docker-compose and one for\nbuilding an optimized production image: [deployment/ci/Dockerfile](./deployment/ci/Dockerfile).\n\n### Versioning\n\nThere is one global exported variable named `Version` in `main.go`. The variable is set \nto the latest tag or commit hash during the build process. You can look in the production \nDockerfile to see how the Version is set during build. The version is printed in the service \nlog on startup and can be used to verify which specific commit of the code is deployed.\n\n\u003e Version should *not* be set or modified manually in the source code.\n\n### Logging\n\nThe service outputs all logs to `stdout` as defined by the best practices in the Cloud Native\ncommunity. See here for more details [12 Factor App](https://12factor.net/logs).\nFrom there logs could be processed as needed in the specific running environment.\nThe standard log levels are `[debug,info,warn,error,fatal`] and `info` is the default level.\nIf you want to set another log level, use the ENV configuration variable `LOG_LEVEL` to set it.\n\n### GDPR\n\n[GDPR](GDPR.md)\n\n### Dependencies\n\n[Dependencies](go.mod)\n\n### Deployment\n\n[Helm deployment documentation](deployment/helm/README.md)\n\n### License\n\n[Apache 2.0 license](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feclipse-xfsc%2Fcustom-policy-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feclipse-xfsc%2Fcustom-policy-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feclipse-xfsc%2Fcustom-policy-agent/lists"}