{"id":13701356,"url":"https://github.com/banterfm/graphql-crunch","last_synced_at":"2025-05-04T21:30:48.844Z","repository":{"id":29676703,"uuid":"122444866","full_name":"banterfm/graphql-crunch","owner":"banterfm","description":"Reduces the size of GraphQL responses by consolidating duplicate values","archived":false,"fork":false,"pushed_at":"2023-03-04T04:06:33.000Z","size":2733,"stargazers_count":534,"open_issues_count":7,"forks_count":17,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-14T02:51:28.022Z","etag":null,"topics":["apollo","graphql","performance","relay"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/banterfm.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-22T07:27:32.000Z","updated_at":"2024-07-22T22:02:04.000Z","dependencies_parsed_at":"2024-06-18T17:03:13.192Z","dependency_job_id":"f0bc47df-4185-4371-93d6-00b5765473a0","html_url":"https://github.com/banterfm/graphql-crunch","commit_stats":{"total_commits":94,"total_committers":15,"mean_commits":6.266666666666667,"dds":0.5531914893617021,"last_synced_commit":"519ca99b373c627cc015ae96944ee1d5377b7032"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/banterfm%2Fgraphql-crunch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/banterfm%2Fgraphql-crunch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/banterfm%2Fgraphql-crunch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/banterfm%2Fgraphql-crunch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/banterfm","download_url":"https://codeload.github.com/banterfm/graphql-crunch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252403713,"owners_count":21742420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apollo","graphql","performance","relay"],"created_at":"2024-08-02T20:01:32.393Z","updated_at":"2025-05-04T21:30:46.727Z","avatar_url":"https://github.com/banterfm.png","language":"TypeScript","funding_links":[],"categories":["TypeScript","JavaScript"],"sub_categories":[],"readme":"# graphql-crunch\n\n[![NPM version](http://img.shields.io/npm/v/graphql-crunch.svg?style=flat-square)](https://www.npmjs.org/package/graphql-crunch)\n\nOptimizes JSON responses by minimizing duplication and improving\ncompressibility.\n\nOn [Banter.fm](https://banter.fm), we see a 76% reduction in raw JSON size and\na 30% reduction in gzip'd size. This leads to reduced transfer time and faster\nJSON parsing on mobile.\n\n- [Client support](#client-support)\n- [Installation](#installation)\n- [How does it work?](#how-does-it-work)\n- [Motivation](#motivation)\n- [Example](#example)\n  - [Small Example](#small-example)\n  - [Large Example](#large-example)\n- [Usage](#usage)\n  - [Server-side](#server-side)\n  - [Client-side](#client-side)\n\n## Client support\n\n`graphql-crunch` is client agnostic and can be used anywhere that sends or\nreceives JSON. We provide examples for integration with\n[`apollo-client`](https://github.com/apollographql/apollo-client) as we use\nthis in a GraphQL environment.\n\n## Installation\n\nThis library is distributed on `npm`. In order to add it as a dependency,\nrun the following command:\n\n```sh\n$ npm install graphql-crunch --save\n```\n\nor with [Yarn](https://yarnpkg.com):\n\n```sh\n$ yarn add graphql-crunch\n```\n\n## How does it work?\n\nWe flatten the object hierarchy into an array using a post-order traversal of\nthe object graph. As we traverse we efficiently check if we've come across a\nvalue before, including arrays and objects, and replace it with a reference to\nit's earlier occurence if we've seen it. Values are only ever present in the\narray once.\n\n### Note: Crunching and uncrunching is an entirely lossless process. The final payload exactly matches the original.\n\n## Motivation\n\nLarge JSON blobs can be slow to parse on some mobile platforms, especially\nolder Android phones, so we set out to improve that. At the same time we also\nwound up making the payloads more amenable to gzip compression too. GraphQL and\nREST-ful API responses tend to have a lot of duplication leading to huge\npayload sizes.\n\n## Example\n\nIn these examples, we use the [SWAPI GraphQL\ndemo](http://graphql.org/swapi-graphql).\n\n### Small Example\n\nUsing this\n[query](\u003chttp://graphql.org/swapi-graphql/?query=%7B%0A%20%20allPeople(first%3A%202)%20%7B%0A%20%20%20%20people%20%7B%0A%20%20%20%20%20%20name%0A%20%20%20%20%20%20gender%0A%20%20%20%20%20%20filmConnection(first%3A%202)%20%7B%0A%20%20%20%20%20%20%20%20films%20%7B%0A%20%20%20%20%20%20%20%20%20%20title%0A%20%20%20%20%20%20%20%20%20%20characterConnection(first%3A%202)%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20characters%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20name%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20gender%0A%20%20%20%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D\u0026operationName=null\u003e)\nwe'll fetch the first 2 people and their first 2 films and the first 2\ncharacters in each of those films. We limit the connections to the first two\nitems to keep the payload small:\n\n```graphql\n{\n  allPeople(first: 2) {\n    people {\n      name\n      gender\n      filmConnection(first: 2) {\n        films {\n          title\n          characterConnection(first: 2) {\n            characters {\n              name\n              gender\n            }\n          }\n        }\n      }\n    }\n  }\n}\n```\n\nWe get this response:\n\n```json\n{\n  \"data\": {\n    \"allPeople\": {\n      \"people\": [\n        {\n          \"name\": \"Luke Skywalker\",\n          \"gender\": \"male\",\n          \"filmConnection\": {\n            \"films\": [\n              {\n                \"title\": \"A New Hope\",\n                \"characterConnection\": {\n                  \"characters\": [\n                    {\n                      \"name\": \"Luke Skywalker\",\n                      \"gender\": \"male\"\n                    },\n                    {\n                      \"name\": \"C-3PO\",\n                      \"gender\": \"n/a\"\n                    }\n                  ]\n                }\n              },\n              {\n                \"title\": \"The Empire Strikes Back\",\n                \"characterConnection\": {\n                  \"characters\": [\n                    {\n                      \"name\": \"Luke Skywalker\",\n                      \"gender\": \"male\"\n                    },\n                    {\n                      \"name\": \"C-3PO\",\n                      \"gender\": \"n/a\"\n                    }\n                  ]\n                }\n              }\n            ]\n          }\n        },\n        {\n          \"name\": \"C-3PO\",\n          \"gender\": \"n/a\",\n          \"filmConnection\": {\n            \"films\": [\n              {\n                \"title\": \"A New Hope\",\n                \"characterConnection\": {\n                  \"characters\": [\n                    {\n                      \"name\": \"Luke Skywalker\",\n                      \"gender\": \"male\"\n                    },\n                    {\n                      \"name\": \"C-3PO\",\n                      \"gender\": \"n/a\"\n                    }\n                  ]\n                }\n              },\n              {\n                \"title\": \"The Empire Strikes Back\",\n                \"characterConnection\": {\n                  \"characters\": [\n                    {\n                      \"name\": \"Luke Skywalker\",\n                      \"gender\": \"male\"\n                    },\n                    {\n                      \"name\": \"C-3PO\",\n                      \"gender\": \"n/a\"\n                    }\n                  ]\n                }\n              }\n            ]\n          }\n        }\n      ]\n    }\n  }\n}\n```\n\nAfter we crunch it, we get:\n\n```json\n{\n  \"data\": [\n    \"male\",\n    \"Luke Skywalker\",\n    { \"gender\": 0, \"name\": 1 },\n    \"n/a\",\n    \"C-3PO\",\n    { \"gender\": 3, \"name\": 4 },\n    [2, 5],\n    { \"characters\": 6 },\n    \"A New Hope\",\n    { \"characterConnection\": 7, \"title\": 8 },\n    \"The Empire Strikes Back\",\n    { \"characterConnection\": 7, \"title\": 10 },\n    [9, 11],\n    { \"films\": 12 },\n    { \"filmConnection\": 13, \"gender\": 0, \"name\": 1 },\n    { \"filmConnection\": 13, \"gender\": 3, \"name\": 4 },\n    [14, 15],\n    { \"people\": 16 },\n    { \"allPeople\": 17 }\n  ]\n}\n```\n\nThe transformed payload is substantially smaller. After converting both\npayloads to JSON (with formatting removed), the transformed payload is 49%\nfewer bytes.\n\nWhen the client receives this, we simply uncrunch it and get back the exact\noriginal version for the client to handle.\n\n### Large Example\n\nIn real-world scenarios, we'll have modularized our shcema with fragments and have\nas well as connections that have more than two items in them. Here's a [query](https://graphql.org/swapi-graphql/?query=%7B%0A%20%20allPeople%20%7B%0A%20%20%20%20people%20%7B%0A%20%20%20%20%20%20...PersonFragment%0A%20%20%20%20%20%20filmConnection%20%7B%0A%20%20%20%20%20%20%20%20films%20%7B%0A%20%20%20%20%20%20%20%20%20%20...FilmFragment%0A%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A%0Afragment%20PersonFragment%20on%20Person%20%7B%0A%20%20name%0A%20%20birthYear%0A%20%20eyeColor%0A%20%20gender%0A%20%20hairColor%0A%20%20height%0A%20%20mass%0A%20%20skinColor%0A%20%20homeworld%20%7B%0A%20%20%20%20name%0A%20%20%20%20population%0A%20%20%7D%0A%7D%0A%0Afragment%20FilmFragment%20on%20Film%20%7B%0A%20%20title%0A%20%20characterConnection%20%7B%0A%20%20%20%20characters%20%7B%0A%20%20%20%20%20%20...PersonFragment%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D%0A)\nsimilar to the one above except we don't limit the size of the connections and\nwe request a standard set of selections on `Person` objects.\n\n```graphql\n{\n  allPeople {\n    people {\n      ...PersonFragment\n      filmConnection {\n        films {\n          ...FilmFragment\n        }\n      }\n    }\n  }\n}\n\nfragment PersonFragment on Person {\n  name\n  birthYear\n  eyeColor\n  gender\n  hairColor\n  height\n  mass\n  skinColor\n  homeworld {\n    name\n    population\n  }\n}\n\nfragment FilmFragment on Film {\n  title\n  characterConnection {\n    characters {\n      ...PersonFragment\n    }\n  }\n}\n```\n\nThe resulting response from this query is roughly 1MB of JSON (989,946 bytes),\nbut with tons of duplication. Here is how crunching impacts the payload size:\n\n|             | Raw      | Crunched | Improvement |\n| ----------- | -------- | -------- | ----------- |\n| Size        | 989,946B | 28,220B  | 97.1%       |\n| GZip'd Size | 22,240B  | 5,069B   | 77.2%       |\n\nThis is an admittedly extreme result, but highlights the potential for\ncrunching payloads with large amounts of duplication.\n\n## Usage\n\n### Server-side\n\nWith [`apollo-server`](https://github.com/apollographql/apollo-server) you\ncan supply a custom `formatResponse` function. We use this to crunch the `data`\nfield of the `response` before sending it over the wire.\n\n```js\nimport { ApolloServer } from \"apollo-server\";\n\nconst server = new ApolloServer({\n  // schema, context, etc...\n  formatResponse: (response) =\u003e {\n    if (response.data) {\n      response.data = crunch(response.data);\n    }\n    return response;\n  },\n});\n\nserver.listen({ port: 80 });\n```\n\nTo maintain compatibility with clients that aren't expecting crunched payloads,\nwe recommend conditioning the crunch on a query param, like so:\n\n```js\nimport url from \"url\";\nimport querystring from \"querystring\";\nimport { ApolloServer } from \"apollo-server\";\n\nconst server = new ApolloServer({\n  // schema, context, etc...\n  formatResponse: (response, options) =\u003e {\n    const parsed = url.parse(options.context.request.url);\n    const query = querystring.parse(parsed.query);\n\n    if (query.crunch \u0026\u0026 response.data) {\n      const version = parseInt(query.crunch) || 1;\n      response.data = crunch(response.data, version);\n    }\n\n    return response;\n  },\n});\n\nserver.listen({ port: 80 });\n```\n\nNow only clients that opt-in to crunched payloads via the `?crunch=2` query\nparameter will receive them.\n\nYour client can specify the version of the crunch format to use in the query\nparameter. If the version isn't specified, or an unknown version is supplied,\nwe default to v1.0.\n\n### Client-side\n\nOn the client, we uncrunch the server response before the GraphQL client\nprocesses it.\n\nWith [`apollo-client`](https://github.com/apollographql/apollo-client), use a\n[`link`](https://www.apollographql.com/docs/react/reference/index.html#types)\nconfiguration to setup an\n[afterware](https://www.apollographql.com/docs/react/basics/network-layer.html#linkAfterware),\ne.g.\n\n```js\nimport { ApolloClient } from 'apollo-client';\nimport { ApolloLink, concat } from 'apollo-link';\nimport { HttpLink } from 'apollo-link-http';\nimport { uncrunch } from 'graphql-crunch';\n\nconst http = new HttpLink({\n  credentials: 'include',\n  uri: '/api'\n});\n\nconst uncruncher = new ApolloLink((operation, forward) =\u003e\n  forward(operation)\n    .map((response) =\u003e {\n      response.data = uncrunch(response.data);\n      return response;\n    });\n);\n\nconst client = new ApolloClient({link: concat(uncruncher, http)});\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbanterfm%2Fgraphql-crunch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbanterfm%2Fgraphql-crunch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbanterfm%2Fgraphql-crunch/lists"}