{"id":26522579,"url":"https://github.com/miracum/vfps","last_synced_at":"2026-03-15T03:27:41.901Z","repository":{"id":60363815,"uuid":"541322652","full_name":"miracum/vfps","owner":"miracum","description":"A very fast and resource-efficient pseudonym service.","archived":false,"fork":false,"pushed_at":"2024-05-22T15:07:48.000Z","size":631,"stargazers_count":4,"open_issues_count":5,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-05-22T19:19:56.753Z","etag":null,"topics":["grpc","grpc-dotnet","grpc-json-transcoder","pseudonym","pseudonymization"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miracum.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-25T22:07:20.000Z","updated_at":"2024-05-28T14:21:08.349Z","dependencies_parsed_at":"2023-10-14T15:27:03.951Z","dependency_job_id":"82f91130-21da-4774-8922-1a7c4a8f9ca1","html_url":"https://github.com/miracum/vfps","commit_stats":null,"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracum%2Fvfps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracum%2Fvfps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracum%2Fvfps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracum%2Fvfps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miracum","download_url":"https://codeload.github.com/miracum/vfps/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244806384,"owners_count":20513434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grpc","grpc-dotnet","grpc-json-transcoder","pseudonym","pseudonymization"],"created_at":"2025-03-21T13:36:43.677Z","updated_at":"2026-03-15T03:27:41.893Z","avatar_url":"https://github.com/miracum.png","language":"C#","readme":"# vfps\n\n![Latest Version](https://img.shields.io/github/v/release/miracum/vfps)\n![License](https://img.shields.io/github/license/miracum/vfps)\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/miracum/vfps/badge)](https://api.securityscorecards.dev/projects/github.com/miracum/vfps)\n[![SLSA 3](https://slsa.dev/images/gh-badge-level3.svg)](https://slsa.dev)\n\nA [very fast](#e2e-load-testing) and [resource-efficient](#resource-efficiency) pseudonym service.\n\nSupports horizontal service replication for highly-available deployments.\n\n## Run it\n\n\u003e **Warning**\n\u003e Using the provided docker-compose.yaml is not a production-ready deployment but merely\n\u003e used to get started and testing quickly.\n\u003e It sets very restrictive resource limits uses the default password for an included,\n\u003e unoptimized PostgreSQL deployment.\n\n```sh\ndocker compose -f docker-compose.yaml --profile=test up\n```\n\nVisit \u003chttp://localhost:8080/\u003e to view the OpenAPI specification of the Vfps API:\n\n![Screenshot of the OpenAPI specification](docs/img/openapi.png)\n\nYou can use the JSON-transcoded REST API described via OpenAPI or interact with the service using gRPC.\nFor example, using [grpcurl](https://github.com/fullstorydev/grpcurl) to create a new namespace:\n\n```sh\ngrpcurl \\\n  -plaintext \\\n  -import-path src/Vfps/ \\\n  -proto src/Vfps/Protos/vfps/api/v1/namespaces.proto \\\n  -d '{\"name\": \"test\", \"pseudonymGenerationMethod\": \"PSEUDONYM_GENERATION_METHOD_SECURE_RANDOM_BASE64URL_ENCODED\", \"pseudonymLength\": 32}' \\\n  127.0.0.1:8081 \\\n  vfps.api.v1.NamespaceService/Create\n```\n\nAnd to create a new pseudonym inside this namespace:\n\n```sh\ngrpcurl \\\n  -plaintext \\\n  -import-path src/Vfps/ \\\n  -proto src/Vfps/Protos/vfps/api/v1/pseudonyms.proto \\\n  -d '{\"namespace\": \"test\", \"originalValue\": \"to be pseudonymized\"}' \\\n  127.0.0.1:8081 \\\n  vfps.api.v1.PseudonymService/Create\n```\n\n## Production-grade deployment\n\nSee \u003chttps://github.com/miracum/charts/tree/master/charts/vfps\u003e for a production-grade deployment on Kubernetes via Helm.\n\n## Configuration\n\nAvailable configuration options which can be set as environment variables:\n\n| Variable                                           | Type         | Default             | Description                                                                                                                   |\n| -------------------------------------------------- | ------------ | ------------------- | ----------------------------------------------------------------------------------------------------------------------------- |\n| `ConnectionStrings__PostgreSQL`                    | `string`     | `\"\"`                | Connection string to the PostgreSQL database. See \u003chttps://www.npgsql.org/doc/connection-string-parameters.html\u003e for options. |\n| `ForceRunDatabaseMigrations`                       | `bool`       | `false`             | Run database migrations as part of the startup. Only recommended when a single replica of the application is used.            |\n| `Tracing__IsEnabled`                               | `bool`       | `false`             | Enable distributed tracing support.                                                                                           |\n| `Tracing__Exporter`                                | `string`     | `\"jaeger\"`          | The tracing export format. One of `jaeger`, `otlp`.                                                                           |\n| `Tracing__ServiceName`                             | `string`     | `\"vfps\"`            | Tracing service name.                                                                                                         |\n| `Tracing__RootSampler`                             | `string`     | `\"AlwaysOnSampler\"` | Tracing parent root sampler. One of `AlwaysOnSampler`, `AlwaysOffSampler`, `TraceIdRatioBasedSampler`                         |\n| `Tracing__SamplingProbability`                     | `double`     | `0.1`               | Sampling probability to use if `Tracing__RootSampler` is set to `TraceIdRatioBasedSampler`.                                   |\n| `Tracing__Jaeger`                                  | `object`     | `{}`                | Jaeger exporter options.                                                                                                      |\n| `Tracing__Otlp__Endpoint`                          | `string`     | `\"\"`                | The OTLP gRPC Endpoint URL.                                                                                                   |\n| `Pseudonymization__Caching__Namespaces__IsEnabled` | `bool`       | `false`             | Set to `true` to enable namespace caching.                                                                                    |\n| `Pseudonymization__Caching__Pseudonyms__IsEnabled` | `bool`       | `false`             | Set to `true` to enable pseudonym caching.                                                                                    |\n| `Pseudonymization__Caching__SizeLimit`             | `int`        | `65534`             | Maximum number of entries in the cache. The cache is shared between the pseudonyms and namespaces.                            |\n| `Pseudonymization__Caching__AbsoluteExpiration`    | `D.HH:mm:nn` | `0.01:00:00`        | Time after which a cache entry expires.                                                                                       |\n\n## Observability\n\nThe service exports metrics in Prometheus format on `:8082/metrics`.\nHealth-, readiness-, and liveness-probes are exposed at `:8080/healthz`, `:8080/readyz`, and `:8080/livez` respectively.\n\n## FHIR operations\n\nThe service also exposes a FHIR operations endpoint. Sending a FHIR Parameters resource to `/v1/fhir/$create-pseudonym`\nof the following schema:\n\n```json\n{\n  \"resourceType\": \"Parameters\",\n  \"parameter\": [\n    {\n      \"name\": \"namespace\",\n      \"valueString\": \"test\"\n    },\n    {\n      \"name\": \"originalValue\",\n      \"valueString\": \"hello world\"\n    }\n  ]\n}\n```\n\nwill create a pseudonym in the `test` namespace. The expected response looks as follows:\n\n```json\n{\n  \"resourceType\": \"Parameters\",\n  \"parameter\": [\n    {\n      \"name\": \"namespace\",\n      \"valueString\": \"test\"\n    },\n    {\n      \"name\": \"originalValue\",\n      \"valueString\": \"hello world\"\n    },\n    {\n      \"name\": \"pseudonymValue\",\n      \"valueString\": \"8KWwnm3TXR5R9iUDVVKD-jUezE4DEyeydOeq4v_a_b5ejSLmqOlT8g\"\n    }\n  ]\n}\n```\n\n## Development\n\n### Prerequisites\n\n- .NET 7.0: \u003chttps://dotnet.microsoft.com/en-us/download/dotnet\u003e\n- Docker CLI 20.10.17: \u003chttps://www.docker.com/\u003e\n- Docker Compose: \u003chttps://docs.docker.com/compose/install/\u003e\n\n### Build \u0026 run\n\nStart an empty PostgreSQL database for development (optionally add `-d` to run in the background):\n\n```sh\ndocker compose -f docker-compose.yaml up\n```\n\nTo additionally start an instance of [Jaeger Tracing](https://www.jaegertracing.io/), you can specify the `jaeger`\nprofile:\n\n```sh\ndocker compose -f docker-compose.yaml --profile=jaeger up\n```\n\nRestore dependencies and run in Debug mode:\n\n```sh\ndotnet restore\ndotnet run -c Debug --project=src/Vfps\n```\n\nOpen \u003chttps://localhost:8080/\u003e to see the OpenAPI UI for the JSON-transcoded gRPC services.\nYou can also use [grpcurl](https://github.com/fullstorydev/grpcurl) to interact with the API:\n\n\u003e **Note**\n\u003e In development mode gRPC reflection is enabled and used by grpcurl by default.\n\n```sh\ngrpcurl -plaintext \\\n    -d '{\"name\": \"test\", \"pseudonymGenerationMethod\": \"PSEUDONYM_GENERATION_METHOD_SECURE_RANDOM_BASE64URL_ENCODED\", \"pseudonymLength\": 32}' \\\n    127.0.0.1:8081 \\\n    vfps.api.v1.NamespaceService/Create\n\ngrpcurl -plaintext \\\n    -d '{\"namespace\": \"test\", \"originalValue\": \"a test value\"}' \\\n    127.0.0.1:8081 \\\n    vfps.api.v1.PseudonymService/Create\n```\n\n#### Run unit tests\n\n```sh\ndotnet test src/Vfps.Tests \\\n  --configuration=Release \\\n  --collect:\"XPlat Code Coverage\" \\\n  --results-directory=./coverage \\\n  -l \"console;verbosity=detailed\" \\\n  --settings=src/Vfps.Tests/runsettings.xml\n```\n\n#### Generate Code coverage report\n\nIf not installed, install the report generation too:\n\n```sh\ndotnet tool install -g dotnet-reportgenerator-globaltool\n```\n\n```sh\nreportgenerator -reports:\"./coverage/*/coverage.cobertura.xml\" -targetdir:\"coveragereport\" -reporttypes:Html\n# remove the coverage directory so successive runs won't cause issues with their random GUID.\n# See \u003chttps://github.com/microsoft/vstest/issues/2378\u003e\nrm -rf coverage/\n```\n\n### Build container image\n\n```sh\ndocker build -t ghcr.io/miracum/vfps:latest .\n```\n\n### Run iter8 SLO experiments locally\n\n```sh\nkind create cluster\n\nexport IMAGE_TAG=\"iter8-test\"\n\ndocker build -t ghcr.io/miracum/vfps:${IMAGE_TAG} .\n\nkind load docker-image ghcr.io/miracum/vfps:${IMAGE_TAG}\n\nhelm upgrade --install \\\n  --set=\"image.tag=${IMAGE_TAG}\" \\\n  -f tests/iter8/values.yaml \\\n  --wait \\\n  --timeout=15m \\\n  --version=^1.0.0 \\\n  vfps oci://ghcr.io/miracum/charts/vfps\n\nkubectl apply -f tests/iter8/experiment.yaml\n\niter8 k assert -c completed --timeout 15m\niter8 k assert -c nofailure,slos\niter8 k report\n```\n\n## Benchmarks\n\n### Micro benchmarks\n\nThe pseudonym generation methods are continuously benchmarked. Results are viewable at \u003chttps://miracum.github.io/vfps/dev/bench/\u003e.\n\n### E2E load testing\n\nCreate a pseudonym namespace used for benchmarking:\n\n```sh\ngrpcurl \\\n  -plaintext \\\n  -import-path src/Vfps/ \\\n  -proto src/Vfps/Protos/vfps/api/v1/namespaces.proto \\\n  -d '{\"name\": \"benchmark\", \"pseudonymGenerationMethod\": \"PSEUDONYM_GENERATION_METHOD_SECURE_RANDOM_BASE64URL_ENCODED\", \"pseudonymLength\": 32}' \\\n  127.0.0.1:8081 \\\n  vfps.api.v1.NamespaceService/Create\n```\n\nGenerate 100.000 pseudonyms in the namespace from random original values:\n\n```sh\nghz -n 100000 \\\n    --insecure \\\n    --import-paths src/Vfps/ \\\n    --proto src/Vfps/Protos/vfps/api/v1/pseudonyms.proto \\\n    --call vfps.api.v1.PseudonymService/Create \\\n    -d '{\"originalValue\": \"{{randomString 32}}\", \"namespace\": \"benchmark\"}' \\\n    127.0.0.1:8081\n```\n\nSample output running on\n\n```console\nOS=Windows 11 (10.0.22000.978/21H2)\n12th Gen Intel Core i9-12900K, 1 CPU, 24 logical and 16 physical cores\n32GiB of DDR4 4800MHz RAM\nSamsung SSD 980 Pro 1TiB\nPostgreSQL running in WSL2 VM on the same machine.\n.NET SDK=7.0.100-rc.1.22431.12\n```\n\n```console\nSummary:\n  Count:        100000\n  Total:        16.68 s\n  Slowest:      187.81 ms\n  Fastest:      2.52 ms\n  Average:      8.00 ms\n  Requests/sec: 5993.51\n\nResponse time histogram:\n  2.522   [1]     |\n  21.051  [99748] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  39.580  [201]   |\n  58.109  [0]     |\n  76.639  [0]     |\n  95.168  [0]     |\n  113.697 [0]     |\n  132.226 [0]     |\n  150.755 [0]     |\n  169.285 [0]     |\n  187.814 [50]    |\n\nLatency distribution:\n  10 % in 6.26 ms\n  25 % in 6.91 ms\n  50 % in 7.72 ms\n  75 % in 8.93 ms\n  90 % in 9.57 ms\n  95 % in 10.01 ms\n  99 % in 11.86 ms\n\nStatus code distribution:\n  [OK]   100000 responses\n```\n\n### Sub-10ms P99-latency\n\nBy default, each pseudonym creation requests executes two database queries: one to fetch the namespace configuration\nand a second one to persist the pseudonym if it doesn't already exist. There is an opt-in way to avoid the first\nquery by caching the namespaces in a non-distributed in-memory cache. It can be enabled and configured using the following\nenvironment variables:\n\n| Variable                                           | Type         | Default      | Description                                |\n| -------------------------------------------------- | ------------ | ------------ | ------------------------------------------ |\n| `Pseudonymization__Caching__Namespaces__IsEnabled` | `bool`       | `false`      | Set to `true` to enable namespace caching. |\n| `Pseudonymization__Caching__SizeLimit`             | `int`        | `32`         | Maximum number of entries in the cache.    |\n| `Pseudonymization__Caching__AbsoluteExpiration`    | `D.HH:mm:nn` | `0.01:00:00` | Time after which a cache entry expires.    |\n\n\u003e **Warning**\n\u003e Deleting a namespace does not automatically remove it from the in-memory cache.\n\u003e Pseudonym creation requests against such a stale cached namespace will fail until\n\u003e either the entry expired or the service is restarted.\n\nUsing the same setup as above but with namespace caching enabled, we can lower the per-request latencies and increase throughput:\n\n```console\nSummary:\n  Count:        100000\n  Total:        11.70 s\n  Slowest:      27.23 ms\n  Fastest:      1.55 ms\n  Average:      5.47 ms\n  Requests/sec: 8549.22\n\nResponse time histogram:\n  1.546  [1]     |\n  4.114  [17418] |∎∎∎∎∎∎∎∎∎∎\n  6.682  [72827] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  9.251  [9382]  |∎∎∎∎∎\n  11.819 [122]   |\n  14.387 [0]     |\n  16.956 [0]     |\n  19.524 [0]     |\n  22.092 [0]     |\n  24.661 [49]    |\n  27.229 [201]   |\n\nLatency distribution:\n  10 % in 4.00 ms\n  25 % in 4.34 ms\n  50 % in 5.81 ms\n  75 % in 6.00 ms\n  90 % in 6.66 ms\n  95 % in 7.00 ms\n  99 % in 8.00 ms\n\nStatus code distribution:\n  [OK]   100000 responses\n```\n\n### Resource efficiency\n\nThe sample deployment described in [docker-compose.yaml](docker-compose.yaml) sets strict resource\nlimits for both the CPU (1 CPU) and memory (max 128MiB). Even under these constraints \u003e 1k RPS are\npossible, although with significantly increased P99 latencies:\n\n```console\nSummary:\n  Count:        100000\n  Total:        73.99 s\n  Slowest:      268.06 ms\n  Fastest:      5.26 ms\n  Average:      36.69 ms\n  Requests/sec: 1351.51\n\nResponse time histogram:\n  5.257   [1]     |\n  31.537  [57298] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  57.817  [21327] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎\n  84.097  [17685] |∎∎∎∎∎∎∎∎∎∎∎∎\n  110.377 [3395]  |∎∎\n  136.656 [243]   |\n  162.936 [0]     |\n  189.216 [1]     |\n  215.496 [0]     |\n  241.776 [0]     |\n  268.055 [50]    |\n\nLatency distribution:\n  10 % in 14.62 ms\n  25 % in 18.47 ms\n  50 % in 29.46 ms\n  75 % in 47.53 ms\n  90 % in 71.96 ms\n  95 % in 79.95 ms\n  99 % in 97.22 ms\n\nStatus code distribution:\n  [OK]   100000 responses\n```\n\n## Image signature and provenance verification\n\nPrerequisites:\n\n- [cosign](https://github.com/sigstore/cosign/releases)\n- [slsa-verifier](https://github.com/slsa-framework/slsa-verifier/releases)\n- [crane](https://github.com/google/go-containerregistry/releases)\n\nAll released container images are signed using [cosign](https://github.com/sigstore/cosign) and SLSA Level 3 provenance is available for verification.\n\n\u003c!-- x-release-please-start-version --\u003e\n\n```sh\nIMAGE=ghcr.io/miracum/vfps:v1.3.7\nDIGEST=$(crane digest \"${IMAGE}\")\nIMAGE_DIGEST_PINNED=\"ghcr.io/miracum/vfps@${DIGEST}\"\nIMAGE_TAG=\"${IMAGE#*:}\"\n\ncosign verify \\\n   --certificate-oidc-issuer=https://token.actions.githubusercontent.com \\\n   --certificate-identity-regexp=\"https://github.com/miracum/.github/.github/workflows/standard-build.yaml@.*\" \\\n   --certificate-github-workflow-name=\"ci\" \\\n   --certificate-github-workflow-repository=\"miracum/vfps\" \\\n   --certificate-github-workflow-trigger=\"release\" \\\n   --certificate-github-workflow-ref=\"refs/tags/${IMAGE_TAG}\" \\\n   \"${IMAGE_DIGEST_PINNED}\"\n\nslsa-verifier verify-image \\\n    --source-uri github.com/miracum/vfps \\\n    --source-tag ${IMAGE_TAG} \\\n    \"${IMAGE_DIGEST_PINNED}\"\n```\n\nSee also \u003chttps://github.com/slsa-framework/slsa-github-generator/tree/main/internal/builders/container#verification\u003e for details on verifying the image integrity using automated policy controllers.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiracum%2Fvfps","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiracum%2Fvfps","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiracum%2Fvfps/lists"}