{"id":21298345,"url":"https://github.com/yornaath/batshit","last_synced_at":"2025-04-04T16:11:33.989Z","repository":{"id":65152747,"uuid":"581157423","full_name":"yornaath/batshit","owner":"yornaath","description":"A batch manager that will deduplicate and batch requests for a certain data type made within a window. Useful to batch requests made from multiple react components that uses react-query","archived":false,"fork":false,"pushed_at":"2024-06-04T12:16:03.000Z","size":64825,"stargazers_count":162,"open_issues_count":4,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-28T15:06:41.709Z","etag":null,"topics":["async","batch-processing","concurrency","deduplication","fetch","react","react-query","tanstack","typescript"],"latest_commit_sha":null,"homepage":"https://batshit-example.vercel.app/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yornaath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-22T12:29:10.000Z","updated_at":"2025-03-27T13:22:52.000Z","dependencies_parsed_at":"2024-05-20T20:11:48.206Z","dependency_job_id":"22a78d35-be33-41e3-805f-63c26567a1fb","html_url":"https://github.com/yornaath/batshit","commit_stats":{"total_commits":155,"total_committers":1,"mean_commits":155.0,"dds":0.0,"last_synced_commit":"d5b17714f98f6c18bc6db55083c2acdb2c21a30f"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yornaath%2Fbatshit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yornaath%2Fbatshit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yornaath%2Fbatshit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yornaath%2Fbatshit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yornaath","download_url":"https://codeload.github.com/yornaath/batshit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247208142,"owners_count":20901570,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","batch-processing","concurrency","deduplication","fetch","react","react-query","tanstack","typescript"],"created_at":"2024-11-21T14:50:51.994Z","updated_at":"2025-04-04T16:11:33.950Z","avatar_url":"https://github.com/yornaath.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# @yornaath/batshit [![CI](https://github.com/yornaath/batshit/actions/workflows/ci.yml/badge.svg)](https://github.com/yornaath/batshit/actions/workflows/ci.yml)\n\nA batch manager that will deduplicate and batch requests for a given data type made within a window of time (or other custom scheduling). Useful to batch requests made from multiple react components that uses react-query or do batch processing of accumulated tasks.\n\n### Codesandbox example\nHere is a codesanbox example using react, typescript, vite and the zeitgeist prediction-markets indexer api.\nIt fetches markets up front and then batches all liquidity pool fetches made from the individual components into one request.\n\n[Codesandbox](https://codesandbox.io/s/yornaath-batshit-example-8f8q3w?file=/src/App.tsx)\n\n### Example with devtools\nExample using zeitgeist market and pool data with included devtools to inspect the batching process.\nThe working live code for the example linked below can be found in [./packages/example](https://github.com/yornaath/batshit/tree/master/packages/example)\n\n[Vercel Example app](https://batshit-example.vercel.app/)\n\n## Install\n```bash\nyarn add @yornaath/batshit\n```\n\n## Quickstart\n\nHere we are creating a simple batcher that will batch all fetches made within a window of 10 ms into one request.\n\n```ts\nimport { create, keyResolver, windowScheduler } from \"@yornaath/batshit\";\n\ntype User = { id: number; name: string };\n\nconst users = create({\n  fetcher: async (ids: number[]) =\u003e {\n    return client.users.where({\n      id_in: ids,\n    });\n  },\n  resolver: keyResolver(\"id\"),\n  scheduler: windowScheduler(10), // Default and can be omitted.\n});\n\n/**\n * Requests will be batched to one call since they are done within the same time window of 10 ms.\n */\nconst bob = users.fetch(1);\nconst alice = users.fetch(2);\n\nconst bobUndtAlice = await Promise.all([bob, alice]);\n\nawait delay(100);\n\n/**\n * New Requests will be batched in a another call since not within the first timeframe.\n */\nconst joe = users.fetch(3);\nconst margareth = users.fetch(4);\n\nconst joeUndtMargareth = await Promise.all([joe, margareth]);\n```\n\n## React(query) Example\n\nHere we are also creating a simple batcher that will batch all fetches made within a window of 10 ms into one request. Since all \u003cUserDetails /\u003e items are rendered in one go their individual fetches will be batched into one request.\n\n**Note: a batcher for a group of items should only be created once. So creating them inside hooks wont work as intended.**\n\n```ts\nimport { useQuery } from \"react-query\";\nimport { create, windowScheduler } from \"@yornaath/batshit\";\n\nconst users = create({\n  fetcher: async (ids: number[]) =\u003e {\n    return client.users.where({\n      userId_in: ids,\n    });\n  },\n  resolver: keyResolver(\"id\"),\n  scheduler: windowScheduler(10),\n});\n\nconst useUser = (id: number) =\u003e {\n  return useQuery([\"users\", id], async () =\u003e {\n    return users.fetch(id);\n  });\n};\n\nconst UserDetails = (props: { userId: number }) =\u003e {\n  const { isFetching, data } = useUser(props.userId);\n  return (\n    \u003c\u003e\n      {isFetching ? (\n        \u003cdiv\u003eLoading user {props.userId}\u003c/div\u003e\n      ) : (\n        \u003cdiv\u003eUser: {data.name}\u003c/div\u003e\n      )}\n    \u003c/\u003e\n  );\n};\n\n/**\n * Since all user details items are rendered within the window there will only be one request made.\n */\nconst UserList = () =\u003e {\n  const userIds = [1, 2, 3, 4];\n  return (\n    \u003c\u003e\n      {userIds.map((id) =\u003e (\n        \u003cUserDetails userId={id} /\u003e\n      ))}\n    \u003c/\u003e\n  );\n};\n```\n\n### Limit batch size\n\nWe provide two helper functions for limiting the number of batched fetch calls.\n\n#### `windowedFiniteBatchScheduler`\n\nThis will batch all calls made within a certain time frame UP to a certain max batch size before it starts a new batch\n\n```ts\nconst batcher = batshit.create({\n  ...,\n  scheduler: windowedFiniteBatchScheduler({\n    windowMs: 10,\n    maxBatchSize: 100,\n  }),\n});\n```\n\n#### `maxBatchSizeScheduler`\n\nSame as the one above, but will only wait indefinetly until the batch size is met.\n\n```ts\nconst batcher = batshit.create({\n  ...,\n  scheduler: maxBatchSizeScheduler({\n    maxBatchSize: 100,\n  }),\n});\n```\n\n### Fetching where response is an object of items\n\nIn this example the response is an object/record with the id of the user as the key and the user object as the value.\n\n**Example:**\n```json\n{\n  \"1\": {\"username\": \"bob\"},\n  \"2\": {\"username\": \"alice\"}\n}\n```\n\n```ts\nimport * as batshit from \"@yornaath/batshit\";\n\nconst batcher = batshit.create({\n  fetcher: async (ids: string[]) =\u003e {\n    const users: Record\u003cstring, User\u003e = await fetchUserRecords(ids)\n    return users\n  },\n  resolver: batshit.indexedResolver(),\n});\n```\n\n### Fetching with needed context\n\nIf the batch fetcher needs some context like an sdk or client to make its fetching you can use a memoizer to make sure that you reuse a batcher for the given context in the hook calls.\n\n```ts\nimport { useQuery } from \"@tanstack/react-query\";\nimport { memoize } from \"lodash-es\";\nimport * as batshit from \"@yornaath/batshit\";\n\nexport const key = \"markets\";\n\nconst batcher = memoize((sdk: Sdk\u003cIndexerContext\u003e) =\u003e {\n  return batshit.create({\n    name: key,\n    fetcher: async (ids: number[]) =\u003e {\n      const { markets } = await sdk.markets({\n        where: {\n          marketId_in: ids,\n        },\n      });\n      return markets;\n    },\n    scheduler: batshit.windowScheduler(10),\n    resolver: batshit.keyResolver(\"marketId\"),\n  });\n});\n\nexport const useMarket = (marketId: number) =\u003e {\n  const [sdk, id] = useSdk();\n\n  const query = useQuery(\n    [id, key, marketId],\n    async () =\u003e {\n      if(sdk) {\n        return batcher(sdk).fetch(marketId);\n      }\n    },\n    {\n      enabled: Boolean(sdk),\n    },\n  );\n\n  return query;\n};\n```\n\n# Custom Batch Resolver\n\nThis batcher will fetch all posts for multiple users in one request and resolve the correct list of posts for the discrete queries.\n\n```ts\nconst userposts = create({\n  fetcher: async (queries: { authorId: number }) =\u003e {\n    return api.posts.where({\n      authorId_in: queries.map((q) =\u003e q.authorId),\n    });\n  },\n  scheduler: windowScheduler(10),\n  resolver: (posts, query) =\u003e\n    posts.filter((post) =\u003e post.authorId === query.authorId),\n});\n\nconst [alicesPosts, bobsPost] = await Promise.all([\n  userposts.fetch({authorId: 1})\n  userposts.fetch({authorId: 2})\n]);\n```\n\n# React Devtools\n\nTools to debug and inspect the batching process can be found in the [@yornaath/batshit-devtools-react](https://www.npmjs.com/package/@yornaath/batshit-devtools-react) package.\n\n```bash\nyarn add @yornaath/batshit-devtools @yornaath/batshit-devtools-react\n```\n\n```ts\nimport { create, keyResolver, windowScheduler } from \"@yornaath/batshit\";\nimport BatshitDevtools from \"@yornaath/batshit-devtools-react\";\n\nconst batcher = create({\n  fetcher: async (queries: number[]) =\u003e {...},\n  scheduler: windowScheduler(10),\n  resolver: keyResolver(\"id\"),\n  name: \"batcher:data\" // used in the devtools to identify a particular batcher.\n});\n\nconst App = () =\u003e {\n  \u003cdiv\u003e\n    \u003cBatshitDevtools /\u003e\n  \u003c/div\u003e\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyornaath%2Fbatshit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyornaath%2Fbatshit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyornaath%2Fbatshit/lists"}