https://github.com/zachbateman/easy_multip
Simplify Python multiprocessing capabilities.
https://github.com/zachbateman/easy_multip
multiprocessing python
Last synced: about 1 year ago
JSON representation
Simplify Python multiprocessing capabilities.
- Host: GitHub
- URL: https://github.com/zachbateman/easy_multip
- Owner: zachbateman
- License: mit
- Created: 2019-01-15T19:37:46.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2024-05-03T19:42:15.000Z (about 2 years ago)
- Last Synced: 2025-01-28T21:35:32.412Z (over 1 year ago)
- Topics: multiprocessing, python
- Language: Python
- Homepage:
- Size: 23.4 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# easy_multip
easy_multip is a small tool designed to simplify Python multiprocessing capabilities.
# Current Features
- easy_multip.map(func, arg_group, leave_one_cpu_free=True)
- map() or list comprehension type functionality that is parallelized using multiprocessing and includes a progress bar
- Usage is similar to the below constructs:
```sh
list(map(func, arg_group))
or
[func(arg) for arg in arg_group]
```
- easy_multip.doloop(func, arg_group, leave_one_cpu_free=True)
- for loop equivalent that runs a function that returns None
- Useful in situations like file processing where each operation is expensive and totally independent
- Allocates jobs evenly among processors and provides a progress bar... of sorts
- Usage is similar to the below construct:
```sh
for arg in arg_group:
func(arg)
```
- easy_multip.decorators.use_multip(func, leave_one_cpu_free=True)
- Decorator providing capability of quickly adding multiprocessing to a function operating on a list
- ONLY for functions taking a list first arg that returns a list or None
- DO NOT USE `@decorator` syntax! Must use the following pattern:
```sh
def _func(list_arg, *args, **kwargs):
# stuff happens
return [] (or None)
func = use_multip(_func)
```
License
----
MIT