{"id":20832863,"url":"https://github.com/udhos/gateboard","last_synced_at":"2025-08-18T17:12:39.129Z","repository":{"id":63867489,"uuid":"570315605","full_name":"udhos/gateboard","owner":"udhos","description":"gateboard resolves AWS Private API Gateway ID.","archived":false,"fork":false,"pushed_at":"2025-05-07T22:19:33.000Z","size":1258,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-15T11:50:48.533Z","etag":null,"topics":["api","api-gateway","aws","aws-api-gateway","go","golang"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/udhos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-11-24T21:58:17.000Z","updated_at":"2025-05-07T22:19:33.000Z","dependencies_parsed_at":"2024-01-15T23:19:24.339Z","dependency_job_id":"ec765470-b080-40a9-998b-68fa4342f76d","html_url":"https://github.com/udhos/gateboard","commit_stats":null,"previous_names":[],"tags_count":288,"template":false,"template_full_name":null,"purl":"pkg:github/udhos/gateboard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fgateboard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fgateboard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fgateboard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fgateboard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/udhos","download_url":"https://codeload.github.com/udhos/gateboard/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/udhos%2Fgateboard/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271027687,"owners_count":24687082,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","api-gateway","aws","aws-api-gateway","go","golang"],"created_at":"2024-11-18T00:13:27.976Z","updated_at":"2025-08-18T17:12:39.107Z","avatar_url":"https://github.com/udhos.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![license](http://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/udhos/gateboard/blob/main/LICENSE)\n[![Go Report Card](https://goreportcard.com/badge/github.com/udhos/gateboard)](https://goreportcard.com/report/github.com/udhos/gateboard)\n[![Go Reference](https://pkg.go.dev/badge/github.com/udhos/gateboard.svg)](https://pkg.go.dev/github.com/udhos/gateboard)\n[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/gateboard)](https://artifacthub.io/packages/search?repo=gateboard)\n[![Docker Pulls gateboard](https://img.shields.io/docker/pulls/udhos/gateboard)](https://hub.docker.com/r/udhos/gateboard)\n[![Docker Pulls gateboard-discovery](https://img.shields.io/docker/pulls/udhos/gateboard-discovery)](https://hub.docker.com/r/udhos/gateboard-discovery)\n\n# gateboard\n\n[gateboard](https://github.com/udhos/gateboard) resolves AWS Private API Gateway ID.\n\nIn order to [invoke an AWS API Gateway private API](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-private-api-test-invoke-url.html), an applications needs its API ID.\n\nHowever, whenever API is destroyed and recreated, the API ID will change.\n\nThis projects helps in building an autodiscovery mechanism to withstand changes in the API ID.\n\nThere are three main pieces:\n\n1\\. `gateboard` service ([gateboard](./cmd/gateboard)) provides a simple REST API for saving/retrieving name-to-id mappings:\n\n```bash\n# add api1=\u003eid1 mapping\ncurl -X PUT -d '{\"gateway_id\":\"id1\"}' localhost:8080/gateway/api1\n\n# retrieve mapping for api1\ncurl localhost:8080/gateway/api1\n{\"gateway_name\":\"api1\",\"gateway_id\":\"id1\"}\n```\n\n2\\. `gateboard-discovery` service ([gateboard-discovery](./cmd/gateboard-discovery)) provides autodiscovery by periodically scanning AWS API Gateway API and updating `gateboard` with discovered name-to-id mappings.\n\n3\\. `gateboard` client ([gateboard](./gateboard)) is a helper library to assist client application in retrieving current API ID for a given API name. The main benefit from this client is automatic caching for the name-to-id mappings.\n\n`gateboard` client usage looks like this:\n\n```golang\nclient := gateboard.NewClient(gateboard.ClientOptions{\n    ServerURL: \"http://gateboard:8080/gateway\",\n})\n\napiName := \"my-api\"\n\napiID := client.GatewayID(ctx, apiName)\nif gatewayID == \"\" {\n    log.Printf(\"missing gateway_id for gateway_name=%s\", apiName)\n    return\n}\n\n// code to invoke AWS API with apiID follows\n```\n\nFind client documentation here: https://pkg.go.dev/github.com/udhos/gateboard@main/gateboard\n\n# Features\n\n- [X] SQS listener\n- [X] Client with async update\n- [X] Create mongodb index on startup\n- [X] Define TTL on server record (60s), restrict acceptable TTL range on client (60s..600s)\n- [X] Repository tests\n- [X] HTTP server tests\n- [X] SQS tests\n- [X] Docker image\n- [X] Client tests\n- [X] Refactor config\n- [X] Repository DynamoDB\n- [X] Optional authentication\n- [X] AWS Secrets Manager\n- [X] Preload token for optional authentication\n- [X] Gateway load sharing\n- [X] Repository redis\n- [X] Discovery service\n- [X] Metrics\n- [X] Repository S3\n- [X] Tracing\n- [X] Benchmark\n- [ ] User guide\n- [X] Zap logging\n- [ ] Cache service\n- [X] Multiple repositories - basic tests\n- [X] Multiple repositories - dump\n- [X] Multiple repositories - multirepo tests\n- [X] Multiple repositories - README\n- [X] Multiple repositories - secrets\n- [X] Multiple repositories - metrics\n- [X] Multiple repositories - helm chart\n- [X] Multiple repositories - fastest concurrent response\n- [X] Multiple repositories - fastest concurrent response tests\n- [X] Group cache\n\n# Services\n\n| Service  | Description |\n| --- | --- |\n| gateboard | Holds database of key value mappings: gateway_name =\u003e gateway_id. You can populate the database as you wish. |\n| gateboard-discovery | Can be used to scan AWS API Gateway APIs and to save the name x ID mappings into gateboard. |\n| gateboard-cache | Can be used as local fast cache to save resources on a centralized main gateboard service. |\n\n# Build\n\n```bash\ngit clone https://github.com/udhos/gateboard\ncd gateboard\nCGO_ENABLED=0 go install ./...\n```\n\n# Supported Repositories (Persistent Storage)\n\n```\n# Available repo kinds:\n#\n# mem:      testing-only pseudo-storage\n# mongo:    MongoDB\n# redis:    redis\n# dynamodb: DynamoDB\n# s3:       S3\n\nexport REPO_LIST=repo.yaml\n\n$ cat repo.yaml\n- kind: mem\n  name: mem1 # name is used for metrics\n- kind: mem\n  name: mem2 # pick distinct names for multiple instances of the same kind\n\n- kind: mongo\n  name: mongo1 # name is used for metrics\n  mongo:\n    uri: mongodb://localhost:27017/\n    database: gateboard\n    collection: gateboard\n    username: \"\"\n    #password: \"aws-parameterstore:us-east-1:/mongo/cluster1/password\" # see https://github.com/udhos/boilerplate\n    #tls_ca_file: /etc/gateboard/mongo-tls-ca-bundle.pem\n    min_pool: 1\n    index_creation_disable: false\n    index_creation_retry: 5\n    index_creation_cooldown: 5s\n\n- kind: dynamodb\n  name: dynamo1 # name is used for metrics\n  dynamodb:\n    table: gateboard\n    region: us-east-1\n    role_arn: \"\"\n    manual_create: false # if false, gateboard will create the table automatically\n\n- kind: redis\n  name: redis1 # name is used for metrics\n  redis:\n    addr: localhost:6379\n    #password: \"aws-parameterstore:us-east-1:/redis/cluster3/password\" # see https://github.com/udhos/boilerplate\n    key: gateboard\n    #tls: true\n    #tls_insecure_skip_verify: true\n    client_name: auto # 'auto' means use hostname\n\n- kind: s3\n  name: s3one # name is used for metrics\n  s3:\n    bucket_name: \"\"\n    bucket_region: us-east-1\n    prefix: gateboard\n    role_arn: \"\"\n    manual_create: false # if false, gateboard will create the bucket automatically\n    #server_side_encryption: AES256\n```\n\n# Testing repositories\n\n## Testing repository mongo\n\nStart mongodb:\n\n```bash\ndocker run --rm --name mongo-main -p 27017:27017 mongo\n```\n\nRun repository tests:\n\n```bash\nexport TEST_REPO_MONGO=true ;# enable mongodb tests\ngo test -count=1 -run TestRepository ./cmd/gateboard\n```\n\n## Testing repository dynamodb\n\nCreate a dynamodb table named `gateboard_test` with partition key `gateway_name`.\n\nMake sure the table is empty before running the tests.\n\nRun repository tests:\n\n```bash\nexport TEST_REPO_DYNAMO=true ;# enable dynamodb tests\ngo test -count=1 -run TestRepository ./cmd/gateboard\n```\n\n## Testing repository redis\n\nStart redis:\n\n```bash\ndocker run --rm --name redis-main -p 6379:6379 -d redis\n```\n\nRun repository tests:\n\n```bash\nexport TEST_REPO_REDIS=true ;# enable redis tests\ngo test -count=1 -run TestRepository ./cmd/gateboard\n```\n\n## Testing repository S3\n\nCreate a bucket.\n\nMake sure the bucket is empty before running the tests.\n\nRun repository tests:\n\n```bash\nexport TEST_REPO_S3=put_bucket_name_here ;# enable S3 tests\ngo test -count=1 -run TestRepository ./cmd/gateboard\n```\n\n## Optional Authentication\n\nEnable `WRITE_TOKEN=true` in order to require token authentication for write requests.\n\n```bash\nexport WRITE_TOKEN=true\n```\n\nMake sure the repository has the token `token1` assigned to gateway `gw2`.\n\n```bash\nExample for mongodb:\n\ndb.gateboard.insertOne({\"gateway_name\":\"gw2\",\"token\":\"token1\"})\n```\n\nNow requests to update gateway `gw2` must include the token `token1`.\n\n\n```bash\ncurl -X PUT -s -d '{\"gateway_id\":\"id1\",\"token\":\"token1\"}' localhost:8080/gateway/gw2\n\n{\"gateway_name\":\"gw2\",\"gateway_id\":\"id1\"}\n```\n\nOtherwise the request will be denied.\n\n```bash\ncurl -X PUT -v -d '{\"gateway_id\":\"id2\"}' localhost:8080/gateway/gw2\n*   Trying ::1:8080...\n* TCP_NODELAY set\n* Connected to localhost (::1) port 8080 (#0)\n\u003e PUT /gateway/gw2 HTTP/1.1\n\u003e Host: localhost:8080\n\u003e User-Agent: curl/7.68.0\n\u003e Accept: */*\n\u003e Content-Length: 20\n\u003e Content-Type: application/x-www-form-urlencoded\n\u003e \n* upload completely sent off: 20 out of 20 bytes\n* Mark bundle as not supporting multiuse\n\u003c HTTP/1.1 401 Unauthorized\n\u003c Content-Type: application/json; charset=utf-8\n\u003c Date: Sat, 17 Dec 2022 00:59:22 GMT\n\u003c Content-Length: 65\n\u003c \n* Connection #0 to host localhost left intact\n{\"gateway_name\":\"gw2\",\"gateway_id\":\"id2\",\"error\":\"invalid token\"}\n```\n\n# Examples\n\n```bash\ncurl localhost:8080/gateway/gate1\n{\"gateway_name\":\"gate1\",\"gateway_id\":\"\",\"error\":\"gatewayGet: not found: repository gateway not found error\"}\n\ncurl -X PUT -d '{\"gateway_id\":\"id1\"}' localhost:8080/gateway/gate1\n{\"gateway_name\":\"gate1\",\"gateway_id\":\"id1\"}\n\ncurl localhost:8080/gateway/gate1\n{\"gateway_name\":\"gate1\",\"gateway_id\":\"id1\"}\n\ncurl -X PUT -d '{\"gateway_id\":\"id2\"}' localhost:8080/gateway/gate1\n{\"gateway_name\":\"gate1\",\"gateway_id\":\"id2\"}\n\ncurl localhost:8080/gateway/gate1\n{\"gateway_name\":\"gate1\",\"gateway_id\":\"id2\"}\n```\n\n# gateway-discovery\n\n## Save to server\n\nDiscovery writes directly to server.\n\nStart server.\n\n    export REPO_LIST=repo.yaml\n    cat repo.yaml\n    - kind: mem\n      name: mem1\n    gateboard\n\nRun discovery.\n\n    export SAVE=server\n    export DRY_RUN=false\n    gateboard-discovery\n\nDump database.\n\n    curl localhost:8080/dump | jq\n\n## Save to webhook\n\nDiscovery writes to webhook that forwards to SQS queue.\n\nStart server.\n\n    export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard\n    export REPO_LIST=repo.yaml\n    cat repo.yaml\n    - kind: mem\n      name: mem1\n    gateboard\n\nRun discovery.\n\n    export SAVE=webhook\n    # use lambda function url as webhook\n    export WEBHOOK_URL=https://xxxxxxxxxxxxx.lambda-url.us-east-1.on.aws\n    export DRY_RUN=false\n    gateboard-discovery\n\nDump database.\n\n    curl localhost:8080/dump | jq\n\n## Save to SQS\n\nDiscovery writes to SQS queue.\n\nStart server.\n\n    export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard\n    export REPO_LIST=repo.yaml\n    cat repo.yaml\n    - kind: mem\n      name: mem1\n    gateboard\n\nRun discovery.\n\n    export SAVE=sqs\n    export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard\n    export DRY_RUN=false\n    gateboard-discovery\n\nDump database.\n\n    curl localhost:8080/dump | jq\n\n## Save to SNS\n\nDiscovery writes to SNS topic that forwards to SQS queue.\n\nStart server.\n\n    export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard\n    export REPO_LIST=repo.yaml\n    cat repo.yaml\n    - kind: mem\n      name: mem1\n    gateboard\n\nRun discovery.\n\n    export SAVE=sns\n    export TOPIC_ARN=arn:aws:sns:us-east-1:123456789012:gateboard\n    export DRY_RUN=false\n    gateboard-discovery\n\nDump database.\n\n    curl localhost:8080/dump | jq\n\n## Save to lambda\n\nDiscovery writes to lambda function that forwards to SQS queue.\n\nStart server.\n\n    export QUEUE_URL=https://sqs.us-east-1.amazonaws.com/123456789012/gateboard\n    export REPO_LIST=repo.yaml\n    cat repo.yaml\n    - kind: mem\n      name: mem1\n    gateboard\n\nRun discovery.\n\n    export SAVE=lambda\n    export LAMBDA_ARN=arn:aws:lambda:us-east-1:123456789012:function:forward_to_sqs\n    export DRY_RUN=false\n    gateboard-discovery\n\nDump database.\n\n    curl localhost:8080/dump | jq\n\n# AWS Secrets Manager\n\nRetrieve config vars from AWS Secrets Manager.\n\n    export CONFIG_VAR=aws-secretsmanager:region:name:json_field\n\nSee detailed documentation at https://github.com/udhos/boilerplate.\n\nExample:\n\n    export MONGO_URL=aws-secretsmanager::mongo_uri\n\n    # The secret `mongo_uri` must store a scalar value like: `mongodb://127.0.0.1:27017`\n\nExample with JSON field `uri`:\n\n    export MONGO_URL=aws-secretsmanager::mongo:uri\n\n    # The secret `mongo` must store a JSON value like: `{\"uri\":\"mongodb://127.0.0.2:27017\"}`\n\n# Metrics\n\n```\n# HELP http_server_requests_seconds Spring-like server request duration in seconds.\n# TYPE http_server_requests_seconds histogram\n\nExample: http_server_requests_seconds_bucket{method=\"GET\",status=\"200\",uri=\"/gateway/*gateway_name\",le=\"0.001\"} 4\n\n# HELP repository_requests_seconds Repository request duration in seconds.\n# TYPE repository_requests_seconds histogram\n\nExample: repository_requests_seconds_bucket{method=\"get\",status=\"success\",le=\"0.00025\"} 4\n```\n\n# Test Jaeger Tracing\n\n```\n# start jaeger\n./run-jaeger-local.sh\n\n# start gateboard\nexport REPO_LIST=repo.yaml\ncat repo.yaml\n- kind: mem\n    name: mem1\n\nexport OTEL_TRACES_SAMPLER=parentbased_always_on\nexport OTEL_SERVICE_NAME=mynamespace.gateboard\n# Jaeger\nexport OTELCONFIG_EXPORTER=jaeger\nexport OTEL_TRACES_EXPORTER=jaeger\nexport OTEL_PROPAGATORS=b3multi\nexport OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger-collector:14268\n# OTLP gRPC\n#export OTELCONFIG_EXPORTER=grpc\n#export OTEL_TRACES_EXPORTER=otlp\n#export OTEL_PROPAGATORS=b3multi\n#export OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger-collector:4317\n# OTLP HTTP\n#export OTELCONFIG_EXPORTER=http\n#export OTEL_TRACES_EXPORTER=otlp\n#export OTEL_PROPAGATORS=b3multi\n#export OTEL_EXPORTER_OTLP_ENDPOINT=http://jaeger-collector:4318\n\ngateboard\n\n# open jaeger UI http://localhost:16686/\n```\n\nJaeger: https://www.jaegertracing.io/docs/1.44/getting-started/\n\nOpen Telemetry Go: https://opentelemetry.io/docs/instrumentation/go/getting-started/\n\n\n# Docker\n\n## gateboard docker image\n\nDocker hub:\n\nhttps://hub.docker.com/r/udhos/gateboard\n\nPull from docker hub:\n\n```\ndocker pull udhos/gateboard:0.0.0\n```\n\nBuild recipe:\n\n```\n./docker/build.sh\n```\n\nMultiarch build recipe:\n\n```\n./docker/build-multiarch.sh\n```\n\n## gateboard-discovery docker image\n\nDocker hub:\n\nhttps://hub.docker.com/r/udhos/gateboard-discovery\n\nPull from docker hub:\n\n```\ndocker pull udhos/gateboard-discovery:0.0.0\n```\n\nBuild recipe:\n\n```\n./docker/build-discovery.sh\n```\n\nPush:\n\n```\ndocker push -a udhos/gateboard-discovery\n```\n\n# Helm charts\n\nYou can use the provided helm charts to install gateboard in your Kubernetes cluster.\n\nSee: https://udhos.github.io/gateboard/\n\n# Datadog\n\nImage tags suffixed with `-datadog` are instrumented with Datadog [Orchestrion](https://github.com/DataDog/orchestrion).\n\nSee https://hub.docker.com/r/udhos/gateboard/tags\n\n## Example\n\nInstall latest image instrumented with support for Datadog:\n\n    helm upgrade --install gateboard-dd ./charts/gateboard --set image.tag=latest-datadog\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudhos%2Fgateboard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fudhos%2Fgateboard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fudhos%2Fgateboard/lists"}