{"id":13563014,"url":"https://github.com/apollosolutions/federation-subscription-tools","last_synced_at":"2025-04-03T19:32:07.291Z","repository":{"id":42986613,"uuid":"352712011","full_name":"apollosolutions/federation-subscription-tools","owner":"apollosolutions","description":"A set of demonstration utilities to facilitate GraphQL subscription usage alongside a federated data graph","archived":true,"fork":false,"pushed_at":"2023-08-30T22:09:46.000Z","size":130,"stargazers_count":106,"open_issues_count":11,"forks_count":34,"subscribers_count":13,"default_branch":"main","last_synced_at":"2024-11-04T15:52:13.851Z","etag":null,"topics":["apollo-federation","demo","graphql","subscriptions"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apollosolutions.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-03-29T16:33:03.000Z","updated_at":"2024-06-15T15:45:21.000Z","dependencies_parsed_at":"2023-10-20T17:30:32.774Z","dependency_job_id":null,"html_url":"https://github.com/apollosolutions/federation-subscription-tools","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apollosolutions%2Ffederation-subscription-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apollosolutions%2Ffederation-subscription-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apollosolutions%2Ffederation-subscription-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apollosolutions%2Ffederation-subscription-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apollosolutions","download_url":"https://codeload.github.com/apollosolutions/federation-subscription-tools/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247065359,"owners_count":20877761,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apollo-federation","demo","graphql","subscriptions"],"created_at":"2024-08-01T13:01:14.283Z","updated_at":"2025-04-03T19:32:06.962Z","avatar_url":"https://github.com/apollosolutions.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Using Subscriptions with a Federated Data Graph\n\n## Update June 2023: [Federated subscriptions are now supported in GraphOS](https://www.apollographql.com/blog/announcement/backend/federated-subscriptions-in-graphos-real-time-data-at-scale/)! You can now use subscriptions in your supergraph without these sidecar solutions.\n\nThis demonstration library shows how a decoupled subscription service can run alongside a federated data graph to provide real-time updates to a client. While the subscription service runs a separate non-federated Apollo Server, client applications do not need to perform any special handling of their subscription operations and may send those requests as they would to any GraphQL API that supports subscriptions. The subscription service's API may also specify return types for the `Subscription` fields that are defined in the federated data graph without explicitly redefining them in that service's type definitions.\n\n**In brief, the utilities contained within this library will allow you to:**\n\n- Create a decoupled, independently scalable subscriptions service to run alongside a unified data graph\n- Use types defined in your unified data graph as return types within the subscriptions service's type definitions (without manually redefining those types in the subscriptions service)\n- Publish messages to a shared pub/sub implementation from any subgraph service\n- Allow clients to write subscription operations just as they would if the `Subscription` fields were defined directly within the unified data graph itself\n\n## Example Usage\n\nThe following section outlines how to use the utilities included with this library. The following code is based on a complete working example that has been included in the `example` directory of this repository. Please reference the full example code for additional implementation details and context.\n\n### Make an Executable Schema from Federated and Subscription Type Definitions\n\nThe subscriptions service should only contain a definition for the `Subscription` object type, the types on this field may output any of the types defined in the federated data graph's schema:\n\n```js\n// typeDefs.js (subscriptions service)\n\nimport gql from \"graphql-tag\";\n\nexport const typeDefs = gql`\n  type Subscription {\n    postAdded: Post\n  }\n`;\n```\n\nTo make the federated data graph's types available to the subscription service, instantiate an `ApolloGateway` and call the `makeSubscriptionSchema` function in the gateway's `onSchemaLoadOrUpdate` method to combine its schema with the subscription service's type definitions and resolvers to make the complete executable schema.\n\n**Managed federation option:**\n\n```js\n// index.js (subscriptions service)\nlet schema;\nconst gateway = new ApolloGateway();\n\ngateway.onSchemaLoadOrUpdate(schemaContext =\u003e {\n  schema = makeSubscriptionSchema({\n    gatewaySchema: schemaContext.apiSchema,\n    typeDefs,\n    resolvers\n  });\n});\n\nawait gateway.load({ apollo: getGatewayApolloConfig(apolloKey, graphVariant) });\n```\n\n**Unmanaged federation option:**\n\n```js\n// index.js (subscriptions service)\nlet schema;\nconst gateway = new ApolloGateway({\n  serviceList: [\n    /* Provide your service list here... */\n  ],\n  experimental_pollInterval = 36000;\n});\n\ngateway.onSchemaLoadOrUpdate(schemaContext =\u003e {\n  schema = makeSubscriptionSchema({\n    gatewaySchema: schemaContext.apiSchema,\n    typeDefs,\n    resolvers\n  });\n});\n\nawait gateway.load();\n```\n\nNote that for unmanaged federation, we must set a poll interval to query the subgraph services for their schemas to detect a schema change. Polling the running endpoint for these SDLs is fairly blunt approach, so in production, a more computationally efficient approach would be preferable (or managed federation).\n\n### Use an Apollo Data Source to Fetch Non-Payload Fields\n\nThe subscription service can resolve fields that are included in a published message's payload, but it will need to reach out to the federated data graph to resolve additional non-payload fields. Using an Apollo data source subclassed from the provided `GatewayDataSource`, specific methods can be defined that fetch the non-payload fields by diffing the payload fields with the overall selection set. Optionally, headers (etc.) may be attached to the request to the federated data graph by providing a `willSendRequest` method:\n\n```js\n// LiveBlogDataSource/index.js (subscriptions service)\n\nimport { GatewayDataSource } from \"federation-subscription-tools\";\nimport gql from \"graphql-tag\";\n\nexport class LiveBlogDataSource extends GatewayDataSource {\n  constructor(gatewayUrl) {\n    super(gatewayUrl);\n  }\n\n  willSendRequest(request) {\n    if (!request.headers) {\n      request.headers = {};\n    }\n\n    request.headers[\"apollographql-client-name\"] = \"Subscriptions Service\";\n    request.headers[\"apollographql-client-version\"] = \"0.1.0\";\n\n    // Forwards the encoded token extracted from the `connectionParams` with\n    // the request to the gateway\n    request.headers.authorization = `Bearer ${this.context.token}`;\n  }\n\n  async fetchAndMergeNonPayloadPostData(postID, payload, info) {\n    const selections = this.buildNonPayloadSelections(payload, info);\n    const payloadData = Object.values(payload)[0];\n\n    if (!selections) {\n      return payloadData;\n    }\n\n    const Subscription_GetPost = gql`\n      query Subscription_GetPost($id: ID!) {\n        post(id: $id) {\n          ${selections}\n        }\n      }\n    `;\n\n    try {\n      const response = await this.query(Subscription_GetPost, {\n        variables: { id: postID }\n      });\n      return this.mergeFieldData(payloadData, response.data.post);\n    } catch (error) {\n      console.error(error);\n    }\n  }\n}\n\n```\n\nIn the resolvers for the subscription field, the `fetchAndMergeNonPayloadPostData` method may be called to resolve all requested field data:\n\n```js\n// resolvers.js (subscriptions service)\n\nconst resolvers = {\n  Subscription: {\n    postAdded: {\n      resolve(payload, args, { dataSources: { gatewayApi } }, info) {\n        return gatewayApi.fetchAndMergeNonPayloadPostData(\n          payload.postAdded.id,\n          payload, // known field values\n          info // contains the complete field selection set to diff\n        );\n      },\n      subscribe(_, args) {\n        return pubsub.asyncIterator([\"POST_ADDED\"]);\n      }\n    }\n  }\n};\n```\n\nIn effect, this means that as long the resource that is used as the output type for any subscriptions field may be queried from the federated data graph, then this node may be used as an entry point to that data graph to resolve non-payload fields.\n\nFor the gateway data source to be accessible in `Subscription` field resolvers, we must manually add it to the request context using the `addGatewayDataSourceToSubscriptionContext` function. Note that this example uses [graphql-ws](https://github.com/enisdenjo/graphql-ws) to serve the WebSocket-enabled endpoint for subscription operations. A sample implementation may be structured as follows:\n\n```js\n// index.js (subscriptions service)\n\nconst httpServer = http.createServer(function weServeSocketsOnly(_, res) {\n  res.writeHead(404);\n  res.end();\n});\n\nconst wsServer = new ws.Server({\n  server: httpServer,\n  path: \"/graphql\"\n});\n\nuseServer(\n  {\n    execute,\n    subscribe,\n    context: ctx =\u003e {\n      // If a token was sent for auth purposes, retrieve it here\n      const { token } = ctx.connectionParams;\n\n      // Instantiate and initialize the GatewayDataSource subclass\n      // (data source methods will be accessible on the `gatewayApi` key)\n      const liveBlogDataSource = new LiveBlogDataSource(gatewayEndpoint);\n      const dataSourceContext = addGatewayDataSourceToSubscriptionContext(\n        ctx,\n        liveBlogDataSource\n      );\n\n      // Return the complete context for the request\n      return { token: token || null, ...dataSourceContext };\n    },\n    onSubscribe: (_ctx, msg) =\u003e {\n      // Construct the execution arguments\n      const args = {\n        schema,\n        operationName: msg.payload.operationName,\n        document: parse(msg.payload.query),\n        variableValues: msg.payload.variables\n      };\n\n      const operationAST = getOperationAST(args.document, args.operationName);\n\n      // Stops the subscription and sends an error message\n      if (!operationAST) {\n        return [new GraphQLError(\"Unable to identify operation\")];\n      }\n\n      // Handle mutation and query requests\n      if (operationAST.operation !== \"subscription\") {\n        return [\n          new GraphQLError(\"Only subscription operations are supported\")\n        ];\n      }\n\n      // Validate the operation document\n      const errors = validate(args.schema, args.document);\n\n      if (errors.length \u003e 0) {\n        return errors;\n      }\n\n      // Ready execution arguments\n      return args;\n    }\n  },\n  wsServer\n);\n\nhttpServer.listen({ port }, () =\u003e {\n  console.log(\n    `🚀 Subscriptions ready at ws://localhost:${port}${wsServer.options.path}`\n  );\n});\n```\n\n## Try the Demo\n\n### Installation \u0026 Set-up\n\nThe full example code can be found in the `example` directory. To run the example, you'll need to create a new graph in Apollo Studio for the gateway, [configure rover](https://www.apollographql.com/docs/rover/configuring) with your `APOLLO_KEY`, and then push the two services' schemas:\n\n```sh\nrover subgraph introspect http://localhost:4001 | rover subgraph publish blog@current --schema - --name authors --routing-url http://localhost:4001\n```\n\n```sh\nrover subgraph introspect http://localhost:4002 | rover subgraph publish blog@current --schema - --name posts --routing-url http://localhost:4002\n```\n\n**Important!** The services for the authors and posts subgraphs will need to be running to fetch their schemas from the specified endpoints. You can quickly start up these services without the overhead of running a full `docker-compose` first by running `npm run server:authors` and `npm run server:posts` from the `example/gateway-server` directory (in two different terminal windows). Once the schemas have been successfully pushed to Apollo Studio, you can kill these processes.\n\nNext, add `.env` files to the server and client directories:\n\n1. Add a `.env` file to the `example/gateway-server` directory using the `example/gateway-server/.env.sample` file as a template. Add your new `APOLLO_KEY` and `APOLLO_GRAPH_REF` as variables.\n2. Add a `.env` file to the `example/subscriptions-server` directory using the `example/subscriptions-server/.env.sample` file as a template. Add the same Apollo API key as the `APOLLO_KEY` and `APOLLO_GRAPH_REF`.\n3. Add a `.env` file to the `example/client` directory using the `example/client/.env.sample` file as a template.\n\nFinally, run `docker-compose up --build` from the `example` directory to start all services.\n\nTLDR;\n\n```bash\ncp example/gateway-server/.env.sample example/gateway-server/.env\ncp example/subscriptions-server/.env.sample example/subscriptions-server/.env\ncp example/client/.env.sample example/client/.env\ndocker-compose up --build\n```\n\nThe federated data graph endpoint may be accessed at [http://localhost:4000/graphql](http://localhost:4000/graphql).\n\nThe subscriptions service WebSocket endpoint may be accessed at [ws://localhost:5000/graphql](ws://localhost:5000/graphql).\n\nA React app will be available at [http://localhost:3000](http://localhost:3000).\n\n### Usage\n\nTo see the post list in the client app update in real-time, add a new post at [http://localhost:3000/post/add](http://localhost:3000/post/add) or run the following mutation directly:\n\n```graphql\nmutation AddPost {\n  addPost(authorID: 1, content: \"Hello, world!\", title: \"My Next Post\") {\n    id\n    author {\n      name\n    }\n    content\n    publishedAt\n    title\n  }\n}\n```\n\n### Rationale\n\nThe architecture demonstrated in this project seeks to provide a bridge to native `Subscription` operation support in Apollo Federation. This approach to subscriptions has the advantage of allowing the Apollo Gateway API to remain as the \"stateless execution engine\" of a federated data graph while offloading all subscription requests to a separate service, thus allowing the subscription service to be scaled independently of the gateway.\n\nTo allow the `Subscription` fields to specify return types that are defined in gateway API only, the federated data graph's type definitions are merged with the subscription service's type definitions and resolvers in the gateway's `onSchemaChange` callback to avoid re-declaring these types explicitly here.\n\n### Architectural Details\n\n#### Components\n\nDocker will start five different services with `docker-compose up`:\n\n**1. Gateway Server + Subgraph Services**\n\nThis service contains the federated data graph. For simplicity's sake, two implementing services (for authors and posts) have been bundled with the gateway API in this service. Each implementing service connects to Redis as needed so it can publish events from mutations (the \"pub\" end of subscriptions). For example:\n\n```js\nimport { pubsub } from \"./redis\";\n\nexport const resolvers = {\n  // ...\n  Mutation: {\n    addPost(root, args, context, info) {\n      const post = newPost();\n      pubsub.publish(\"POST_ADDED\", { postAdded: post });\n      return post;\n    }\n  }\n};\n```\n\n**2. Subscriptions Server**\n\nThis service also connects to Redis to facilitate the \"sub\" end of the subscriptions. This service is where the `Subscription` type and related fields are defined. As a best practice, only define a `Subscription` type and applicable resolvers in this service.\n\nWhen sending subscription data to clients, the subscription service can't automatically resolve any data beyond what's provided in the published payload from the implementing service. This means that to resolve nested types (or any other fields that aren't immediately available in the payload object), the resolvers must be defined in the subscription services to fetch this data on a field-by-field basis.\n\nThere are a number of possible approaches that could be taken here, but one recommended approach is to provide an Apollo data source with methods that automatically compare the fields included in the payload against the fields requested in the operation, then selectively query the necessary field data in a single request to the gateway, and finally combine the returned data with the with original payload data to fully resolve the request. For example:\n\n```js\nimport { pubsub } from \"./redis\";\n\nexport const resolvers = {\n  Subscription: {\n    postAdded: {\n      resolve(payload, args, { dataSources: { gatewayApi } }, info) {\n        return gatewayApi.fetchAndMergeNonPayloadPostData(\n          payload.postAdded.id,\n          payload,\n          info\n        );\n      },\n      subscribe(_, args) {\n        return pubsub.asyncIterator([\"POST_ADDED\"]);\n      }\n    }\n  }\n};\n```\n\n**3. Redis**\n\nA shared Redis instance is used to capture publications from the services behind the federated data graph as well as the subscriptions initiated in the subscriptions service, though other [`PubSub` implementations](https://www.apollographql.com/docs/apollo-server/data/subscriptions/#pubsub-implementations) could easily be supported. Note that an in-memory pub/sub implementation will not work because it cannot be shared between the separate gateway and subscription services.\n\n**4. React App**\n\nThe React app contains a homepage with a list of posts as well as a form to add new posts. When a new post is added, the feed of posts on the homepage will be automatically updated.\n\n#### Diagram\n\nThe architecture of the provided example may be visualized as follows:\n\n![Architectural diagram of a federated data graph with a subscriptions service and a React client app](./example/architecture.drawio.svg)\n\n## Important Considerations\n\n**Subscriptions Must be Defined in a Single Service:**\n\nThis solution requires all `Subscription` fields to be defined in a single, decoupled subscription service. This requirement may necessitate that ownership of this service is shared amongst teams that otherwise manage independent portions of the schema applicable to queries and mutations.\n\n**Synchronizing Event Labels:**\n\nSome level of coordination would be necessary to ensure that event labels (e.g. `POST_ADDED`) are synchronized between the implementing services that publish events and the subscription service that calls the `asyncIterator` method with these labels as arguments. Breaking changes may occur without such coordination.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapollosolutions%2Ffederation-subscription-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapollosolutions%2Ffederation-subscription-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapollosolutions%2Ffederation-subscription-tools/lists"}