Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/d-markey/squadron
Multithreading and worker thread pool for Dart / Flutter, to offload CPU-bound and heavy I/O tasks to Isolate or Web Worker threads.
https://github.com/d-markey/squadron
dart isolate parallelism-library thread webworker
Last synced: 5 days ago
JSON representation
Multithreading and worker thread pool for Dart / Flutter, to offload CPU-bound and heavy I/O tasks to Isolate or Web Worker threads.
- Host: GitHub
- URL: https://github.com/d-markey/squadron
- Owner: d-markey
- License: mit
- Created: 2021-09-25T14:45:06.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-11-02T14:30:38.000Z (6 days ago)
- Last Synced: 2024-11-02T15:24:18.674Z (5 days ago)
- Topics: dart, isolate, parallelism-library, thread, webworker
- Language: Dart
- Homepage: https://pub.dev/packages/squadron
- Size: 69.8 MB
- Stars: 85
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
## **Squadron - Multithreading and worker pools in Dart**
Offload CPU-bound and long running tasks and give your apps some air!
Works everywhere: desktop, server, device, browser.
Supports native, JavaScript & Web Assembly platforms.
[](https://pub.dev/packages/squadron)
[](https://pub.dev/packages/squadron)
[](https://pub.dev/packages/squadron)
[](https://github.com/d-markey/squadron/blob/master/LICENSE)
[](https://dart.dev/null-safety)
[](https://pub.dev/packages/lints)
[](https://pub.dev/packages/squadron/score)
[](https://pub.dev/packages/squadron/score)
[](https://pub.dev/packages/squadron/score)
[](https://github.com/d-markey/squadron/commits)
[](https://github.com/d-markey/squadron/actions/workflows/dart.yml)
[](https://github.com/d-markey/squadron/tree/main/coverage/html)
[](https://github.com/d-markey/squadron/tree/main/coverage/html)
[View latest documentation on GitHub](https://github.com/d-markey/squadron/blob/main/README.md)
1. Update your `pubspec.yaml` file to add dependencies to **[Squadron][pub_squadron]** and **[squadron_builder][pub_squadron_builder]** + [build_runner](https://pub.dev/packages/build_runner):
```yaml
dependencies:
squadron: ^6.0.0
# ...
dev_dependencies:
build_runner:
squadron_builder: ^6.0.0
# ...
```
2. Have dart download and install the dependencies:
```bash
dart pub get
```
Create a class containing the code you intend to run in a dedicated thread and make sure you provide `squadron` annotations:
* use **`SquadronService`** for the class;
* use **`SquadronMethod`** for the methods you want to expose.
Service methods must return a `Future`, a `FutureOr` or a `Stream`.
```dart
// file hello_world.dart
import 'dart:async';
import 'package:squadron/squadron.dart';
import 'hello_world.activator.g.dart';
part 'hello_world.worker.g.dart';
@SquadronService(baseUrl: '~/workers', targetPlatform: TargetPlatform.vm | TargetPlatform.web)
// or @SquadronService(baseUrl: '~/workers', targetPlatform: TargetPlatform.all)
base class HelloWorld {
@SquadronMethod()
FutureOr hello([String? name]) {
name = name?.trim() ?? 'World';
return 'Hello, $name!';
}
}
```
## Generating the Worker and WorkerPool code
Have [squadron_builder][pub_squadron_builder] generate the code with the following command line:
```bash
dart run build_runner build
```
This command will create the worker and worker pool from your service: `HelloWorldWorker` and `HelloWorldWorkerPool`.
Workers and worker pools generated by [squadron_builder][pub_squadron_builder] implement the same interface as the original service and proxy all method calls to an instance of the service running in its own thread.
In your program, you can instantiate a `Worker` (or a `WorkerPool` if you need more threads) and use it just as you would use your original service.
**Make sure you stop the workers and pools before exiting** your program. Failure to do so will let your program run forever.
```dart
// file main.dart
import 'package:squadron/squadron.dart';
import 'hello_world.dart';
void main() async {
final worker = HelloWorldWorker();
try {
// Squadron will start the worker for you so you don't have to call worker.start()
final message = await worker.hello();
print(message);
} finally {
// make sure the worker is stopped when the program terminates
worker.stop();
}
}
```
**You must compile your code to JavaScript or Web Assembly** if your app is designed to run in a browser.
```bash
dart compile js ".\src\lib\hello_world.web.g.dart" -o "..\web\workers\hello_world.web.g.dart.js"
dart compile wasm ".\src\lib\hello_world.web.g.dart" -o "..\web\workers\hello_world.web.g.dart.wasm"
```
When compiling to only one of Javascript or Web Assembly, you must make sure your service `@SquadronService()` annotation only references the corresponding `TargetPlatform.js` or `TargetPlatform.wasm`.
You can also compile for both targets: at runtime, Squadron will use the workers matching your app's platform. In that case, make sure your service annotation targets platforms `TargetPlatform.js | TargetPlatform.wasm` or shortcut `TargetPlatform.web`.
There are a few constraints to multithreading in Dart:
* **Dart threads do not share memory**: values passed from one side to the other will typically be cloned. Depending on the implementation, this can impact performance.
* **Service methods arguments and return values need to cross thread-boundaries**: on Web platforms, the Dart runtime delegates this to the browser which is not aware of Dart's type-system. Extra-work is necessary to recover strongly-typed data on the receiving-end.
Data sent through Squadron are handled as `dynamic` types: to recover strong types and guarantee type-safety in your code, Squadron provides `Converter`s to "convert" data on the receiving-end:
* native platforms use a `DirectCastConverter` that simply casts data;
* on Web platforms, objects sent to/from a Web Worker leave Dart's realm when they go through the browser's `postMessage()` function, losing their Dart type in the process. They must therefore re-enter Dart's type-system on the receiving end. Squadron provides a `CastConverter` (converting data as well as items in `List`/`Set`/`Map` objects) and a `NumConverter` (adding special handling for `int`/`double` values) depending on the underlying runtime (JavaScript or Web Assembly).
On native platforms, it is generally safe to not bother about custom types and cloning. The Dart VM will take care of copying data when necessary, optimize data-transfer when possible (eg. `String`s do not require copying), and object types are retained.
There are a few constraints on what type of data can be transferred, please refer to [SendPort.send()](https://api.dart.dev/dart-isolate/SendPort/send.html) documentation for more information.
On native platforms, Squadron uses a default `DirectCastConverter` that simply casts data on the receiving end.
Web platforms have stronger constraints when it comes to transferable objects: for more information, please refer to [Transferable objects](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Transferable_objects) documentation or the [HTML specification for transferable objects](https://html.spec.whatwg.org/multipage/structured-data.html#transferable-objects). There may also be differences between browser flavors and versions.
On Web plaforms, Squadron uses a default `CastConverter` (JavaScript runtime) or `NumConverter` (Web Assembly runtime). One of the key differences between Dart, Web Assembly and JavaScript is number handling: JavaScript only really knows `double` numbers whereas Dart and Web Assembly support `int` and `double` as different types. As a result, on JavaScript platforms, [Dart's `int` is actually a subtype of `double`](https://dart.dev/guides/language/numbers#types-and-type-checking) and special care is required when transfering numbers: on Web Assembly platforms, the receiving-end may receive `int` values as `double` and require a conversion back to `int`.
More importantly, custom-types will require marshaling so they can be transferred across worker boundaries. Squadron is not too opinionated and there are various ways to achieve this: eg. using JSON (together with `json_serializer` for instance), by implementing `marshal()`/`unmarshal()` or `toJson()`/`fromJson()` methods in your data classes, or by using [Squadron marshalers](https://pub.dev/documentation/squadron/latest/squadron/SquadronMarshaler-class.html).
For instance, to transfer a Dart `BigInt` instance:
```dart
class BigIntMarshaler implements GenericMarshaler {
const BigIntMarshaler();
@override dynamic marshal(BigInt data) => data.toString();
@override BigInt unmarshal(dynamic data) => BigInt.parse(data);
}
```
Apply the marshaler by annotating `BigInt` parameters and return values:
```dart
@SquadronService(baseUrl: '~/workers', targetPlatform: TargetPlatform.web)
base class BigIntService {
@SquadronMethod()
@BigIntMarshaler()
FutureOr add(@BigIntMarshaler() BigInt a, @BigIntMarshaler() BigInt b)
=> a + b;
}
```
[squadron_builder][pub_squadron_builder] will implement proper conversion in and out when generating the code for `BigIntServiceWorker`.
As stated in a previous paragraph, code designed to run only on native platforms should not worry about data conversion. Because Squadron native workers share the same code and execute in `Isolate`s running in the same Dart VM, they never leave Dart's type-system. Data sent through Squadron is promoted from `dynamic` back to strong-types by simple cast operations.
On Web platforms, things are different because the data was handed over to the browser which down't know anything about Dart types:
* `bool` and `String`: casting is enough to re-enter Dart's type system (handled by `CastConverter`).
* `int` and `double`: integers may be received as floating points numbers; in JavaScript runtimes, `int`is a subtype of `double` and casting is enough (handled by `CastConverter`); in Web Assembly runtimes, integer values may be received as a `double` and require conversion back to `int` (handled by `NumConverter`).
* `List` and `Map`: these objects are received as `List` and `Map` and item, key and value types are systematically lost. Type-casting is not enough and would always fail with a `TypeError` so additional processing is required from the converter.
* `Set`: these objects are received as `List`, converted to a `List` using the converter, then transformed into a `Set` by calling `list.toSet()`.
To handle `List` and `Map` objects as efficiencly as possible, converters provided by Squadron optimize the process when the item type is a base type that can be handled by a simple cast. Eg. when a service method works with a `List`, it is received/sent as a `List` and will be "promoted" back to `List` by simply calling `list.cast()`. For `Map` objects where `K` and `V` are base types, the received `Map` will be cast back to `Map` with `map.cast()`. In these scenarios, cast operations are deferred until an item is accessed. Dart's static type-safety checks guarantee the cast will succeed.
When lists and maps contain elements that cannot be cast, additional processing is required. **For instance, a `List` object sent to a Web Assembly worker will be received as a `List` containing `double` elements!** Because `int` is not a subtype of `double` on Web Assembly runtimes, `list.cast()` cannot be used.
Under such circumstances, list elements must be processed individually and converted back; eg. `NumConverter` handles this specific example as `list.map(_toInt).toList()` where `_toInt` is a function that returns the input value as an `int` after checking it is effectively an `int` or an integral `double`.
For large collections or complex structures (nested lists/maps), this process may impact performance because 1/ `map()` will iterate over all elements and 2/ `toList()` will create a fresh list to hold the converted elements.
It can be optimized in various ways:
* using `Int32List` and other typed data: under the hood, Squadron converters works with the underlying `ByteBuffer` which guarantees type-safety and hopefully comes with efficient cloning.
* alternatively, it is possible to assign the ambiant converter `Squadron.converter` with a specialized/optimized converter. Squadron provides two additional converters that could be useful:
* `InPlaceConverter`: this implementation wraps around original `List` and `Map` instances to avoid creating new list/map instances. `dynamic` items are immediately converted to their strong-type and stored in the original, dynamic list/map. Read operations will simply cast items when accessed.
* `LazyInPlaceConverter`: this implementation also wraps around original `List` and `Map` instances to avoid creating new instances. Conversion is deferred to when items are effectively accessed: the first read will convert the element and store it in the original instance; subsequent reads will find the element has already been converted and will simply cast and return it.
To activate one of these converters, the ambiant converter can be assigned like so:
```dart
Squadron.converter = LazyInPlaceConverter(Squadron.converter);
```
The original, default converter should be provided to guarantee proper handling of `int`.
Converters only take care of base types (strings, numbers, booleans, lists, maps and sets as well as Dart's typed data). The default behavior for other types (whether they're Dart types such as `BigInt` or `Duration`, or custom types that you or a third-party package implemented) is to simply cast the `dynamic` value to the specified type.
But this will only work on native Dart VM. On browser platforms, custom objects must be serialized when sent and deserialized when received. Squadron provides `SquadronMarshaler` for you to implement your own marshaler:
* `S marshal(T data)`: implement this method to serialize an instance of `T` to something that can be transfered, for instance a `List`;
* `T unmarshal(S data)`: implement this method to deserialize from `S` and back to `T`.
`unmarshal(marchal(obj))` should produce an instance of `T` that is functionaly equivalent to the original instance `obj`.
For instance, given the following class:
```dart
class Car {
Car(this.color, this.price, this.engine);
final String color;
final double price;
final Engine engine;
}
enum Engine { gaz, diesel, electric }
```
A marshaler could be implemented as:
```dart
class CarMarshaler implements SquadronMarshaler {
const CarMarshaler();
List marshal(Car data) =>
[
data.color, // color at index 0
data.price, // price at index 1
data.engine.index, // engine at index 2
];
Car unmarshal(List data) =>
Car(
data[0], // index 0
data[1], // index 1
Engine.values.singleWhere((e) => e.index == data[2]), // index 2
);
}
// for use as an annotation
const carMarshaler = CarMarshaler();
```
[squadron_builder][pub_squadron_builder] will use the marshaler based on annotations provided in your service implementation:
```dart
@SquadronService()
class CarService {
@serviceMethod
@carMarshaler
FutureOr buy(double cash, String color) { /* ... */ }
@serviceMethod
FutureOr sell(@carMarshaler Car car) { /* ... */ }
}
```
Alternatively, if you own the target class, you can also simply annotate it:
```dart
@carMarshaler
class Car {
// ...
}
@SquadronService()
class CarService {
@serviceMethod
FutureOr buy(double cash, String color) { /* ... */ }
@serviceMethod
FutureOr sell(Car car) { /* ... */ }
}
```
If your application is designed to run both on native and Web platforms, it is possible to optimize for VM platforms by providing different marshalers depending on the platform and conditionally import the proper implementation.
```dart
///////////// file car_marshaler.web.dart /////////////
class _CarMarshaler implements SquadronMarshaler {
const CarMarshaler();
List marshal(Car data) => [ /* fields */ ];
Car unmarshal(List data) => Car(/* arguments */);
}
// for use as an annotation
const carMarshaler = _CarMarshaler();
///////////// file car_marshaler.vm.dart /////////////
// for use as an annotation
// IdentityMarshalers do nothing :)
const carMarshaler = IdentityMarshaler();
///////////// file car.dart /////////////
import 'car_marshaler.vm.dart'
if (dart.library.js_interop) 'car_marshaler.web.dart';
@carMarshaler
class Car {
// ...
}
```
* [Saad Ardati](https://github.com/SaadArdati) for his patience and feedback when implementing Squadron into his Flutter application.
* [Martin Fink](https://github.com/martin-robert-fink) for the feedback on Squadron's first `Stream` implementation -- this has resulted in huge progress and a major improvement.
* [Klemen Tusar](https://github.com/techouse) for providing a [sample Chopper JSON decoder](https://hadrien-lejard.gitbook.io/chopper/faq#decoding-json-using-isolates) leveraging Squadron.
* [James O'Leary](https://github.com/jpohhhh) for sponsorship and contribution, very much appreciated.
[pub_squadron]: https://pub.dev/packages/squadron
[pub_squadron_builder]: https://pub.dev/packages/squadron_builder