https://github.com/doxakis/multithreadflow
Run multiple steps on multiple threads with uneven distribution (proof of concept)
https://github.com/doxakis/multithreadflow
Last synced: 9 months ago
JSON representation
Run multiple steps on multiple threads with uneven distribution (proof of concept)
- Host: GitHub
- URL: https://github.com/doxakis/multithreadflow
- Owner: doxakis
- License: mit
- Created: 2017-06-05T03:50:40.000Z (about 9 years ago)
- Default Branch: master
- Last Pushed: 2017-06-05T04:18:56.000Z (about 9 years ago)
- Last Synced: 2025-07-02T03:37:36.587Z (12 months ago)
- Language: C#
- Homepage:
- Size: 8.79 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Run multiple steps on multiple threads with uneven distribution
This is a proof of concept.
# Key points:
- Multiple threads per step
- Try to maximize : MaxItemsPerThread
- There is a queue between each step.
- When a thread is about to start a job, it will remove all the items it will use from the queue.
- When it has finished, it can add one or many items on the next queue.
- The data type of the object can change between steps.
# Why uneven distribution
Suppose you have access to multiple machines (via web api)
But, they don't have the same processor count / RAM / etc.
You can optimize the resources for speed.
# Copyright and license
Code released under the MIT license.