Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hmiladhia/dempipe
A simple Pipe creator
https://github.com/hmiladhia/dempipe
Last synced: 2 days ago
JSON representation
A simple Pipe creator
- Host: GitHub
- URL: https://github.com/hmiladhia/dempipe
- Owner: hmiladhia
- License: mit
- Created: 2020-03-22T15:49:14.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2020-04-02T15:48:43.000Z (over 4 years ago)
- Last Synced: 2024-10-12T16:07:59.827Z (about 1 month ago)
- Language: Python
- Size: 76.2 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DemPipe
## Installation
A simple pip install will do :
```bash
python -m pip install DemPipe
```
## Use
```python
from DemPipe import PipeExecutor, Action
with PipeExecutor() as pipe:
actions = [Action(lambda x: x**2, 2), # returns 4
Action(lambda x: x+3, ctx_in='last_value', ctx_out='my_result_name'), # returns 4+3
Action(lambda x: x*2, ctx_in='my_result_name') # returns 2*7
]
result = pipe.execute(actions)
print('result:', result) # 14
print('result (from context):', pipe.context['last_value']) # 14
print('my_result_name:', pipe.context['my_result_name']) # 7
```
You can also use a Trigger Action for conditional Actions in the pipeline !
```python
from DemPipe import PipeExecutor, Action, Trigger
with PipeExecutor() as pipe:
actions = [Action(lambda x: x**2, 2), # returns 4
Trigger(lambda x: x==3, # returns False -> executes the second action
Action(lambda x: x+3, ctx_in='last_value'), # ignored
Action(lambda x: x+7, ctx_in='last_value'), # returns 11
ctx_in='last_value')
]
result = pipe.execute(actions)
print('result:', result) # 11
```
## Error Handling
In case an error occurs, you can configure an automatic e-mail to be sent with the traceback and the error message :
- Create a config file containing your credentials :
```json
{
"mail": {
"mail_server": "smtp.gmail.com",
"mail_port": 587,
"mail_user": "${os_environ[user]}",
"mail_password": "${os_environ[password]}",
"mail_use_tls": true,
"mail_default_receiver": "${os_environ[receiver]}"
},
"pipe_name": "My Pipe"
}
```
- Specify **config_file** to the **PipeExecutor**
```python
from DemPipe import PipeExecutor, Action
with PipeExecutor(config_file='ConfigsFolder.MyConfig') as pipe:
pipe.execute(Action(lambda x: x/0, 2)) # raises ZeroDivisionException
```
## Other Types of actions
In addition to Trigger and Action there exist other kinds of actions :
- **ContextSetter**: Makes it easier to set context values :
```python
from DemPipe import SimplePipeExecutor, Action, ContextSetter
actions = [ContextSetter(var1=5, var2="test string"),
Action(lambda x: x ** 2, ctx_in='var1')]
with SimplePipeExecutor() as pipe:
print(pipe.execute(actions)) # 25
actions = [ContextSetter(lambda c: {'var1': c['var2'] * 3}, var2=4),
Action(lambda x: x ** 2, ctx_in='var1')]
with SimplePipeExecutor() as pipe:
print(pipe.execute(actions)) # 144
```
- **Procedure**: Same as an action but doesn't update the current context with its return value :
```python
from DemPipe import SimplePipeExecutor, Action, Procedure, ContextSetter
actions = [ContextSetter(last_value=3),
Procedure(lambda x: x**2, ctx_in='last_value'),
Action(lambda x: x + 2, ctx_in='last_value')]
with SimplePipeExecutor() as pipe:
print(pipe.execute(actions)) # 5
```
- **SequentialPipe**: to run code sequentially:
```python
from DemPipe import SimplePipeExecutor, SequentialPipe, Action, Trigger
seq_pipe = SequentialPipe([Action(lambda x: x ** 2, 2),
Action(lambda x: x * 3, 5)])
actions = [Trigger(lambda x: x == 2, seq_pipe, Action(lambda x: x ** 2, 6), 2),
Action(lambda a, b: a + b, 2, ctx_in={'b': 'last_value'})]
with SimplePipeExecutor() as pipe:
print('result:', pipe.execute(actions)) # 17
```
- **ParallelPipe**:
```python
import time
from DemPipe import SimplePipeExecutor, ParallelPipe, Action
t0 = time.time()
actions = ParallelPipe(Action(lambda sec: time.sleep(sec) or sec, 0.2),
Action(lambda sec: time.sleep(sec) or sec, 0.5),
Action(lambda sec: time.sleep(sec) or sec, 0.3),
Action(lambda sec: time.sleep(sec) or sec, 0.2))
with SimplePipeExecutor() as pipe:
print('result:', pipe.execute(actions)) # [0.2, 0.5, 0.3, 0.2]
print('time:', time.time() - t0)
```