{"id":15066428,"url":"https://github.com/hacker1024/batcher.dart","last_synced_at":"2026-02-06T16:05:44.189Z","repository":{"id":56826482,"uuid":"327842810","full_name":"hacker1024/batcher.dart","owner":"hacker1024","description":"A robust future batching solution for Dart.","archived":false,"fork":false,"pushed_at":"2021-03-12T07:43:56.000Z","size":31,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-12T21:37:55.947Z","etag":null,"topics":["batcher","dart","dart-library","dart-package","dartlang","futures"],"latest_commit_sha":null,"homepage":"https://pub.dev/packages/batcher","language":"Dart","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hacker1024.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-08T08:27:58.000Z","updated_at":"2024-10-15T16:16:40.000Z","dependencies_parsed_at":"2022-09-20T21:51:03.747Z","dependency_job_id":null,"html_url":"https://github.com/hacker1024/batcher.dart","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hacker1024/batcher.dart","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hacker1024%2Fbatcher.dart","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hacker1024%2Fbatcher.dart/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hacker1024%2Fbatcher.dart/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hacker1024%2Fbatcher.dart/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hacker1024","download_url":"https://codeload.github.com/hacker1024/batcher.dart/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hacker1024%2Fbatcher.dart/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29167870,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T15:38:29.831Z","status":"ssl_error","status_checked_at":"2026-02-06T15:37:48.592Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batcher","dart","dart-library","dart-package","dartlang","futures"],"created_at":"2024-09-25T01:07:58.639Z","updated_at":"2026-02-06T16:05:44.174Z","avatar_url":"https://github.com/hacker1024.png","language":"Dart","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Batcher\n\nA robust future batching solution.\n\n## Usage\n\n### Installation\n\nAdd the package to your `pubspec.yaml`:\n\n```yaml\ndependencies:\n  ...\n  batcher: ^0.1.0\n```\n\nAnd import it:\n\n```dart\nimport 'package:batcher/batcher.dart';\n```\n\n### Batching\n\nA `FutureBatcher` takes an `Iterable` containing _generator functions_. A\ngenerator function creates a future for the batcher when it needs it. Say, for\nexample, you want to download a list of files in batches with `package:http`:\n\n```dart\n// A predetermined list of file URIs.\nfinal fileUris = \u003cUri\u003e[ /*...*/];\n\n// Use a single client to speed up mass connections.\nfinal client = http.Client();\n\n// Create a list of generators.\nfinal downloadFutureGenerators = [\n  for (final uri in fileUris)\n        () =\u003e http.get(uri).then((response) =\u003e response.body),\n];\n```\n\nNote this is not a list of futures; it's a list of anonymous functions that\n_create_ futures. This allows a batcher to make the HTTP requests in batches.\n\nGenerators can be added to a batcher at any time. If any\nthreads are available, they will start to be called right away; otherwise, they\nwill stay in the queue until a thread is ready.\n\nAs well as requiring future generators, a thread count must also be specified.\nThis thread count is the maximum number of futures that will be resolving at any\ntime. This thread count can change at any time; the new number will be used after\nexcess pending operations complete.\n\nWith these two things, futures can be batched in a few different ways:\n\n#### Future-based batching\n\nIn this way, queued results are delivered via futures returned by `get` functions.\n\n```dart\n// Create a batcher with 16 threads.\nfinal batcher = FutureBatcher\u003cString\u003e(16);\n\n// Add a single generator to the queue.\nfinal Future\u003cString\u003e future =\n    batcher.get(() =\u003e http.get(uri).then((response) =\u003e response.body));\n\n// Or, add multiple generators to the queue.\nfinal List\u003cFuture\u003cString\u003e\u003e futures = batcher.getAll(generators);\n\n// The following can also be used to create a batcher and call getAll in one line.\nfinal futures = generators.batch(16);\n```\n\n#### Stream-based batching\nIf you need to get results from completed futures, but you don't care which\nresult came from which generator, this is an appropriate method. It's more\noptimised than future-based batching for large amounts of work.\n\nThis method utilises the `StreamingFutureBatcher` class.\n\nSay, for example, you wish to pull a list of random fake names from an API,\nwith the `getRandomName` function that returns a `Future\u003cString\u003e`:\n\n```dart\n// Create a list of generators.\nfinal generators = List.filled(100, () =\u003e getRandomName());\n\n// Create a streaming batcher from the generators with 16 threads.\nfinal batcher = StreamingFutureBatcher.from(generators, 16);\n\n// Or, use the extension function:\nfinal batcher = generators.streamBatch(16);\n\n// The results stream can be used like any other.\nawait for (final name in batcher.results) {\n  print(name);\n}\n```\n\nBy default, the batcher will close down after all pending generators are\nresolved. This can be changed with a constructor argument (or by using the\ndefault constructor) to get behaviour that's closer to `FutureBatcher`.\n\nNote that `addAll` is used here; while `getAll` also works, it adds overhead by\ndealing with return values that we don't need.\n\n```dart\n// Create a streaming batcher with 16 threads.\nfinal batcher = StreamingFutureBatcher(16);\n\n// Add the generators.\nbatcher.addAll(generators);\n\n// Do stuff (like adding more)...\n...\n\n// Close when done.\nbatcher.close();\n```\n\n#### Simple batching\n\nThis is the way to go if you don't need to `await` anything or remember which\nresults came from which generator.\n\nThis is useful, for example, in I/O operations like writing files to a cache\nservice where you don't care if they finish completing:\n\n```dart\n// File paths and contents.\nfinal paths = \u003cString\u003e[/*...*/];\nfinal contents = \u003cString\u003e[/*...*/];\n\n// Map the file data to generators.\nfinal generators = List.generate(\n  paths.length,\n      (index) =\u003e () =\u003e File(paths[index]).writeAsString(contents[index]),\n);\n\n// Create a batcher with 16 threads.\nfinal batcher = FutureBatcher\u003cvoid\u003e(16);\n\n// Add the generators.\nbatcher.addAll(generators);\n```\n\n## FAQ\n\n- Q: What can this be useful for?  \n  A:\n  - Accelerating network requests (one at a time, you can be limited by server speeds; all at once with `Future.wait`, you can be blocked by the platform or server)\n  - Filesystem I/O, where there are open file limits\n\n\n- Q: How does this compare to [`package:batching_future`](https://pub.dev/packages/batching_future)?  \n  A: Both packages can do similar things, but they're made for different usecases. `batching_future`\n  is great for unpredictable, rapid API calls, with caching support and time-based batching.\n  This package is made for predictable, ongoing batches, taking fixed-sized lists of generators.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhacker1024%2Fbatcher.dart","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhacker1024%2Fbatcher.dart","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhacker1024%2Fbatcher.dart/lists"}