Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/devopspp/pyliveupdate
A runtime python code manipulation framework for profiling, debugging and bugfixing
https://github.com/devopspp/pyliveupdate
debugging instrumentation monkey-patching patching profiling python zero-downtime
Last synced: 3 months ago
JSON representation
A runtime python code manipulation framework for profiling, debugging and bugfixing
- Host: GitHub
- URL: https://github.com/devopspp/pyliveupdate
- Owner: devopspp
- License: gpl-3.0
- Created: 2020-01-24T06:19:28.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2022-08-22T05:47:00.000Z (about 2 years ago)
- Last Synced: 2024-07-09T07:33:15.401Z (4 months ago)
- Topics: debugging, instrumentation, monkey-patching, patching, profiling, python, zero-downtime
- Language: Python
- Homepage:
- Size: 148 KB
- Stars: 63
- Watchers: 5
- Forks: 9
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-profiling - PyLiveUpdate - Tool to help you modify your running python code without stopping it. (3. Application / Python)
README
# PyLiveUpdate
`PyLiveUpdate` is tool to help you modify your running python code without stopping it.
It is useful for modifying long-running server programs in production with zero downtime.
Some modification scenario includes: inject code to profile the runtime of certain lines of code;
inject code to print out a variable for debugging; apply a patch without restarting.
## Install
```
pip install pyliveupdate
```
## How to use
Put these two lines of code in your server program and run it. Try it out with `example/program1.py`.
```
from pyliveupdatescripts import UpdateStub
UpdateStub().start()
```
Start a controller to modify it!
```
> pylu-controller -l 127.0.0.1:50050
```
Some predefined modification available in the controller
```
> FuncProfiler.profile(['__main__.**', 'module1.**']) # inject execution time profiling code into all functions in __main__ and module1
> LineProfiler.profile('__main__.bar', [11, 12]) # inject time profiling code into certain lines
> FuncDebugger.debug('__main__.bar') # inject code to print out function parameter and return value
> LineDebugger.debug('__main__.bar', [11, 12]) # inject code to print out variables in certain lines
> VarDebugger.debug('__main__.bar', 'b') # inject code to print out all values of a variable in different statements
```
These will print out corresponding profiling and debugging info in the program terminal and in `/tmp/pyliveupdate`.
You can also define your own customized modifications.
## Customized modification
There are in general two kinds of modification: instrument and redefine.
You can define them as following and apply with `patch('patch.py')`. Try it out with `example/patch.py`.
#### Instrument code into existing functions
```
from pyliveupdate.update import Instrument, UpdateManager
def _line_after(a):
print(a)
update = Instrument('__main__.bar',
{('line_after', [12, 14]): _line_after})
UpdateManager.apply_update(update)
```
The code injects a `print(a)` in line 12 and 14 in function `__main__.bar`.
#### Redefine (patch) existing functions
```
from pyliveupdate.update import Redefine, UpdateManager
def new_bar(a):
print('new_bar', a)
update = Redefine('__main__', None, {'__main__.bar':new_bar})
UpdateManager.apply_update(update)
```
The code redefines `__main__.bar` with `new_bar`.
## Revert the modification
PyLiveUpdate also support to revert a modification on the fly:
```
> LU.ls() # list all modification
> LU.revert(1) # revert modifation with id 1
```
## Extended tools
PyLiveUpdate also comes with some handy tools based on it:
### Profiler
Dynamically choose functions or lines to profile and show the result with a flamegraph.
1. Start your program with PyLiveUpdate enabled, like `example/patch.py`.
2. Start a controller `pylu-controller -l 127.0.0.1:50050`
3. Start profiling functions with `FP.profile(['__main__.**', 'module1.**'])` or lines with `LineProfiler.profile('__main__.bar', [11, 12])`.
4. Process the logs with `pylu-processlogs -i /tmp/pyliveupdate.log` in another terminal
It will generated a flamegraph and a summary:
#### Flamegraph
#### Function call summary
The following summary gives in process `4510` thread `5`, `views.results` was called `10` times and each time takes `139 ms`, `views.results` called `manager.all` for `20` times.
```
4510-Thread-5
function hit time/hit (ms)
views.results 10 138.562
-manager.all 20 14.212
-__init__.__hash__ 10 0.035
-manager.get_queryset 20 0.922
-query.__init__ 20 0.616
-query.__init__ 20 0.071
```
## Citation
Please kindly cite our work as follows if you want to refer to it in any writing:
```
@inproceedings{huang2021pylive,
title={$\{$PYLIVE$\}$:$\{$On-the-Fly$\}$ Code Change for Python-based Online Services},
author={Huang, Haochen and Xiang, Chengcheng and Zhong, Li and Zhou, Yuanyuan},
booktitle={2021 USENIX Annual Technical Conference (USENIX ATC 21)},
pages={349--363},
year={2021}
}
```