Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/danreynolds/data_batcher
Data batcher batches and de-dupes data fetched in the same task of the event loop.
https://github.com/danreynolds/data_batcher
batching data flutter
Last synced: 2 days ago
JSON representation
Data batcher batches and de-dupes data fetched in the same task of the event loop.
- Host: GitHub
- URL: https://github.com/danreynolds/data_batcher
- Owner: danReynolds
- License: mit
- Created: 2023-09-15T22:09:01.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-01-19T03:54:54.000Z (10 months ago)
- Last Synced: 2024-01-19T04:36:47.154Z (10 months ago)
- Topics: batching, data, flutter
- Language: Dart
- Homepage: https://pub.dev/packages/data_batcher
- Size: 9.77 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Data Batcher
Data batcher batches and de-dupes data fetched in the same task of the event loop.
```dart
final batcher = DataBatcher(
execute: (ids) async {
print(ids) // ['1', '2', '3', '4']
return Api.fetch(ids);
},
);
batcher.add('1');
batcher.add('2');
batcher.addMany(['2', '3']);
await batcher.add('4');
```
When IDs are added to a batcher, they are grouped by the current cycle of the event loop and scheduled to be executed on a micro-task.
Once executed successfully, the batcher resolves each caller's future with its data as shown below:
```dart
final batcher = DataBatcher(
execute: (ids) async {
return ['a', 'b', 'c'];
},
);
batcher.add('1').then(((resp) => print(resp)); // 'a'
batcher.add('2').then(((resp) => print(resp)); // 'b'
batcher.addMany(['2', '3']).then(((resp) => print(resp)) // ['b', 'c']
```
Data added to a batcher across different ticks of the event loop is broken into separate batches:
```dart
final batcher = DataBatcher(
execute: (ids) async {
print(ids);
// ['1', '2']
// ['3']
return Api.fetch(ids);
},
);
batcher.add('1');
await batcher.add('2');
batcher.add('3');
```
By default, data IDs that are still in-flight from a previous batch which are requested again are not re-fetched:
```dart
final batcher = DataBatcher(
execute: (ids) async {
print(ids);
// ['1', '2']
return Future.delayed(Duration(seconds: 5));
},
);
batcher.add('1');
batcher.add('2');
await Future.delayed(Duration(seconds: 1));
batcher.add('1');
```
The second attempt to request data with ID 1 is de-duped, since it is called while the in-flight request for '1' and '2' has not resolved. The Future returned by the second call to add ID 1 will resolve when the first original batch succeeds and with its returned value for ID 1.
If de-duping of in-flight data is not preferred, the `dedupeInFlight` flag can be set to false:
```dart
final batcher = DataBatcher(
dedupeInFlight: false,
execute: (ids) async {
print(ids);
// ['1', '2']
// ['1']
return Future.delayed(Duration(seconds: 5), () {...});
},
);
batcher.add('1');
batcher.add('2');
await Future.delayed(Duration(seconds: 1));
batcher.add('1');
```
IDs are mapped to data responses using the order of the returned data. If needed, an `idExtractor` can be specified instead in order to associated response data with its matching input ID:
```dart
final batcher = DataBatcher(
idExtractor: (dataModel) => dataModel.id,
execute: (ids) async {
print(ids);
// ['1', '2']
return [DataModel('1'), DataModel('2')]
},
);
batcher.add('1');
batcher.add('2');
```