{"id":13562687,"url":"https://github.com/ccamel/go-graphql-subscription-example","last_synced_at":"2025-10-08T06:03:29.400Z","repository":{"id":35264338,"uuid":"169989792","full_name":"ccamel/go-graphql-subscription-example","owner":"ccamel","description":"☝️ go-graphql subscription over Kafka/Redis/NSQ example ","archived":false,"fork":false,"pushed_at":"2025-04-07T14:30:24.000Z","size":1169,"stargazers_count":42,"open_issues_count":2,"forks_count":5,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-07T15:33:59.879Z","etag":null,"topics":["event-stream","events","go","graphql","graphql-subscriptions","kafka","kafka-consumer","nsq","prometheus","reactivex","redis","websocket"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ccamel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-10T14:58:45.000Z","updated_at":"2025-04-07T14:30:28.000Z","dependencies_parsed_at":"2023-02-19T07:46:04.592Z","dependency_job_id":"719790f3-5bcb-48b6-85e2-80b7a68f5824","html_url":"https://github.com/ccamel/go-graphql-subscription-example","commit_stats":{"total_commits":442,"total_committers":4,"mean_commits":110.5,"dds":0.5633484162895928,"last_synced_commit":"ba21ecb4d5dd5b4d9af2b9726644b495005bbbaa"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccamel%2Fgo-graphql-subscription-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccamel%2Fgo-graphql-subscription-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccamel%2Fgo-graphql-subscription-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccamel%2Fgo-graphql-subscription-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ccamel","download_url":"https://codeload.github.com/ccamel/go-graphql-subscription-example/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248670434,"owners_count":21142904,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["event-stream","events","go","graphql","graphql-subscriptions","kafka","kafka-consumer","nsq","prometheus","reactivex","redis","websocket"],"created_at":"2024-08-01T13:01:11.195Z","updated_at":"2025-10-08T06:03:29.325Z","avatar_url":"https://github.com/ccamel.png","language":"Go","readme":"# go-graphql-subscription-example\n\n[![lint](https://img.shields.io/github/actions/workflow/status/ccamel/go-graphql-subscription-example/lint.yml?label=lint\u0026logo=github)](https://github.com/ccamel/go-graphql-subscription-example/actions/workflows/lint.yml)\n[![build](https://img.shields.io/github/actions/workflow/status/ccamel/go-graphql-subscription-example/build.yml?label=build\u0026logo=github)](https://github.com/ccamel/go-graphql-subscription-example/actions/workflows/build.yml)\n![go](https://img.shields.io/github/go-mod/go-version/ccamel/go-graphql-subscription-example?logo=go)\n[![go-report-card](https://goreportcard.com/badge/github.com/ccamel/go-graphql-subscription-example/master)](https://goreportcard.com/report/github.com/ccamel/go-graphql-subscription-example)\n[![maintainability](https://api.codeclimate.com/v1/badges/67162ec92b2fb97bdb3e/maintainability)](https://codeclimate.com/github/ccamel/go-graphql-subscription-example/maintainability)\n[![quality-gate-status](https://sonarcloud.io/api/project_badges/measure?project=ccamel_go-graphql-subscription-example\u0026metric=alert_status)](https://sonarcloud.io/dashboard?id=ccamel_go-graphql-subscription-example)\n[![lines-of-code](https://sonarcloud.io/api/project_badges/measure?project=ccamel_go-graphql-subscription-example\u0026metric=ncloc)](https://sonarcloud.io/dashboard?id=ccamel_go-graphql-subscription-example)\n[![stackshare](https://img.shields.io/badge/Stackshare-%23ffffff.svg?logo=stackshare\u0026logoColor=0690FA)](https://stackshare.io/ccamel/go-graphql-subscription-example)\n[![git3moji](https://img.shields.io/badge/gitmoji-%20😜%20😍-FFDD67.svg?style=flat-square)](https://gitmoji.carloscuesta.me)\n[![magefile](https://magefile.org/badge.svg)](https://magefile.org)\n[![license](https://img.shields.io/github/license/ccamel/go-graphql-subscription-example.svg?style=flat-square)](https://github.com/ccamel/go-graphql-subscription-example/blob/master/LICENSE)\n[![fossa-status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fccamel%2Fgo-graphql-subscription-example.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Fccamel%2Fgo-graphql-subscription-example?ref=badge_shield)\n\n\u003e Project that demonstrates [graphQL] [subscriptions (over Websocket)](https://github.com/apollographql/subscriptions-transport-ws/blob/v0.9.4/PROTOCOL.md) to consume pre-configured topics from different kinds of\n\u003e stream sources like [Apache Kafka](https://kafka.apache.org/), [redis](https://redis.io/), [NSQ](https://nsq.io)...\n\n## Purpose\n\nThis repository implements a simple service allowing clients to consume messages from a topics/channels through a [graphQL](https://graphql.org/) subscription endpoint.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"doc/overview.png\" title=\"overview\"\u003e\n\u003c/p\u003e\n\nThis particular example demonstrates how to perform basic operations such as:\n\n- serve a [graphiQL](https://github.com/graphql/graphiql) page\n- expose a [Prometheus](https://prometheus.io/) endpoint\n- implement a subscription resolver using WebSocket transport (compliant with [Apollo v0.9.16 protocol](https://github.com/apollographql/subscriptions-transport-ws/blob/v0.9.16/PROTOCOL.md))\n- implement custom [graphQL] _scalars_\n- consumer following kind of stream sources:\n  - [Apache Kafka](https://kafka.apache.org/) -\n        an open-source stream-processing software which aims to provide a unified,\n        high-throughput, low-latency platform for handling real-time data feeds.\n  - [Redis Streams](https://redis.io/) -\n        an open source, in-memory data structure store, message broker with [streaming](https://redis.io/topics/streams-intro) capabilities.\n  - [NSQ](https://nsq.io) -\n        a realtime distributed messaging platform designed to operate at scale.\n\n- process messages using [reactive streams](http://reactivex.io/)\n- filter messages using an expression evaluator\n- ...\n\n## Pre-requisites\n\n **Requires Go 1.14.x** or above, which support Go modules. Read more about them [here](https://github.com/golang/go/wiki/Modules).\n\n## Build\n\nThe project comes with a `magefile.go`, so all the main activities can be performed by [mage](https://magefile.org).\n\n:warning: The source code provided is incomplete - build needs a code generation phase, especially for the embedding of the static resources.\n\nTo build the project, simply invoke the `build` target:\n\n```sh\nmage build\n```\n\nAlternately, the project can be build by [docker](https://www.docker.com/):\n\n```sh\nmage docker\n```\n\nCommand will produce the image `ccamel/go-graphql-subscription-example`.\n\n## How to play with it using Kafka?\n\n### 1. Start Kafka server\n\nAt first, kafka must be started. See [official documentation](https://kafka.apache.org/quickstart) for more.\n\nKafka uses [ZooKeeper](https://zookeeper.apache.org/) so you need to first start a ZooKeeper server if you don't already have one.\n\n```sh\n\u003e bin/zookeeper-server-start.sh config/zookeeper.properties\n```\n\nNow start the Kafka server:\n\n```sh\n\u003e bin/kafka-server-start.sh config/server.properties\n```\n\n### 2. Create topics\n\nFor the purpose of the demo, some topics shall be created. So, let's create 2 topics named `topic-a` and `topic-b`,\nwith a single partition and only one replica:\n\n```sh\n\u003e bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic topic-a\n\u003e bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic topic-b\n```\n\n### 3. Start the GraphQL server\n\nThe configuration is pretty straightforward:\n\n```sh\n\u003e ./go-graphql-subscription-example --help\nUsage:\n  go-graphql-subscription-example [flags]\n\nFlags:\n  -h, --help             help for go-graphql-subscription-example\n      --port uint16      The listening port (default 8000)\n      --source string    The URI of the source to connect to\n      --topics strings   The list of topics/stream names that subscribers can consume (default [foo])\n```\n\nRun the application which exposes the 2 previously created topics to subscribers:\n\n```sh\n\u003e ./go-graphql-subscription-example --topics topic-a,topic-b\n```\n\nAlternately, if the docker image has been previously built, the container can be started this way:\n\n```sh\n\u003e docker run -ti --rm -p 8000:8000 ccamel/go-graphql-subscription-example --topics topic-a,topic-b\n```\n\n### 4. Subscribe\n\nThe application exposes a graphql endpoint through which clients can receive messages coming from a kafka topic.\n\nNavigate to `http://localhost:8000/graphiql` URL and submit the subscription to the topic `topic-a`.\n\n```graphql\nsubscription {\n  event(on: \"topic-a\")\n}\n```\n\nThe offset id to consume from can also be specified. Negative values have a special meaning:\n\n- `-1`: the most recent offset available for a partition (end)\n- `-2`: the least recent offset available for a partition (beginning)\n\n```graphql\nsubscription {\n  event(on: \"topic-a\", at: -1)\n}\n```\n\nAdditionally, a filter expression can be specified. The events consumed are then only ones matching the given predicate.\nYou can refer to [antonmedv/expr] for an overview of the syntax to use to write predicates.\n\n```graphql\nsubscription {\n  event(\n    on: \"topic-a\",\n    at: -1,\n    matching: \"value \u003e 8\"\n  )\n}\n```\n\n### 5. Push messages\n\nRun the producer and then type a few messages into the console to send to Kafka. Note that messages shall be\n[JSON objects](https://www.json.org/).\n\n```sh\n\u003e bin/kafka-console-producer.sh --broker-list localhost:9092 --topic topic-a\n{ \"message\": \"hello world !\", \"value\": 14 }\n```\n\nThe message should be displayed on the browser.\n\n## How to play with it using Redis?\n\n⚠️ Redis implementation does not support offsets (i.e. the capability to resume at some point in time).\n\n### 1. Start Redis\n\nAt first, a redis server (at least v5.0) must be started. See [official documentation](https://redis.io/download) for more.\n\n### 2. Start the GraphQL server\n\nRun the application which exposes the 2 previously created topics to subscribers:\n\n```sh\n\u003e ./go-graphql-subscription-example --source redis://6379?name=foo --topics topic-a,topic-b\n```\n\nAlternately, if the docker image has been previously built, the container can be started this way:\n\n```sh\n\u003e docker run -ti --rm -p 8000:8000 ccamel/go-graphql-subscription-example --source redis://6379?name=foo --topics topic-a,topic-b\n```\n\n### 3. Subscribe\n\nThe application exposes a graphql endpoint through which clients can receive messages coming from a redis stream.\n\nNavigate to `http://localhost:8000/graphiql` URL and submit the subscription to the topic `topic-a`.\n\n```graphql\nsubscription {\n  event(on: \"topic-a\")\n}\n```\n\nAdditionally, a filter expression can be specified. The events consumed are then only ones matching the given predicate.\nYou can refer to [antonmedv/expr] for an overview of the syntax to use to write predicates.\n\n```graphql\nsubscription {\n  event(\n    on: \"topic-a\",\n    matching: \"message contains \\\"hello\\\"\"\n  )\n}\n```\n\n### 4. Push messages\n\nStart the `redis-cli` and then use the `XADD` command to send the messages to the Redis stream.\n\n```sh\n\u003e redis-cli\n127.0.0.1:6379\u003e XADD topic-a * message \"hello world !\" \"value\" \"14\"\n```\n\nThe message should be displayed on the browser.\n\n## How to play with it using NSQ?\n\n⚠️ NSQ implementation does not support offsets (i.e. the capability to resume at some point in time).\n\n### 1. Start NSQ\n\nAt first, NSQ must be started. See [official documentation](https://nsq.io/overview/quick_start.html) for more.\n\n```sh\n\u003e nsqlookupd\n\u003e nsqd --lookupd-tcp-address=127.0.0.1:4160\n\u003e nsqadmin --lookupd-http-address=127.0.0.1:4161\n```\n\n### 2. Start the GraphQL server\n\nRun the application which exposes the 2 previously created topics to subscribers:\n\n```sh\n\u003e ./go-graphql-subscription-example --source nsq: --topics topic-a,topic-b\n```\n\nAlternately, if the docker image has been previously built, the container can be started this way:\n\n```sh\n\u003e docker run -ti --rm -p 8000:8000 ccamel/go-graphql-subscription-example --source nsq: --topics topic-a,topic-b\n```\n\n### 3. Subscribe\n\nThe application exposes a graphql endpoint through which clients can receive messages coming from a redis stream.\n\nNavigate to `http://localhost:8000/graphiql` URL and submit the subscription to the topic `topic-a`.\n\n```graphql\nsubscription {\n  event(on: \"topic-a\")\n}\n```\n\nAdditionally, a filter expression can be specified. The events consumed are then only ones matching the given predicate.\nYou can refer to [antonmedv/expr] for an overview of the syntax to use to write predicates.\n\n```graphql\nsubscription {\n  event(\n    on: \"topic-a\",\n    matching: \"message contains \\\"hello\\\"\"\n  )\n}\n```\n\n### 4. Push messages\n\nPublish a message to the topic `topic-a` by using the command line below:\n\n```sh\n\u003e curl -d '{ \"message\": \"hello world !\", \"value\": 14 }' 'http://127.0.0.1:4151/pub?topic=topic-a'\n```\n\nThe message should be displayed on the browser.\n\n## Stack\n\n### Technical\n\nThis application mainly uses:\n\n- **GraphQL**\n\n    ↳ [graph-gophers/graphql-go](https://github.com/graph-gophers/graphql-go)\n\n    ↳ [graph-gophers/graphql-transport-ws](https://github.com/graph-gophers/graphql-transport-ws)\n\n    ↳ [graphql/graphiql](https://github.com/graphql/graphiql)\n\n- **Prometheus**\n\n    ↳ [prometheus/client_golang](https://github.com/prometheus/client_golang)\n\n- **Kafka**\n\n    ↳ [segment-integrations/connect-kafka](https://github.com/segment-integrations/connect-kafka)\n\n- **Redis**\n\n    ↳ [robinjoseph08/redisqueue](https://github.com/robinjoseph08/redisqueue)\n\n- **NSQ**\n\n    ↳ [nsqio/go-nsq](https://github.com/nsqio/go-nsq)\n\n- **Expression language**\n\n    ↳ [antonmedv/expr]\n\n- **Design Patterns**\n\n    ↳ [ReactiveX/RxGo v2](https://github.com/ReactiveX/RxGo/tree/v2.0.0)\n\n- **CLI**\n\n    ↳ [spf13/cobra](https://github.com/spf13/cobra)\n\n- **Log**\n\n    ↳ [rs/zerolog](https://github.com/rs/zerolog)\n\n### Project\n\n- **Build**\n\n    ↳ [mage](https://magefile.org)\n\n- **Linter**\n\n    ↳ [golangci-lint](https://github.com/golangci/golangci-lint)\n\n## License\n\n[MIT] © [Chris Camel]\n\n[![fossa-status-large](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fccamel%2Fgo-graphql-subscription-example.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fccamel%2Fgo-graphql-subscription-example?ref=badge_large)\n\n[antonmedv/expr]: https://github.com/antonmedv/expr\n\n[graphQL]: https://graphql.org/\n","funding_links":[],"categories":["Implementations","Go"],"sub_categories":["Go"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fccamel%2Fgo-graphql-subscription-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fccamel%2Fgo-graphql-subscription-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fccamel%2Fgo-graphql-subscription-example/lists"}