https://github.com/EverythingMe/probe
Android performance instrumentation tool
https://github.com/EverythingMe/probe
Last synced: 6 months ago
JSON representation
Android performance instrumentation tool
- Host: GitHub
- URL: https://github.com/EverythingMe/probe
- Owner: EverythingMe
- License: other
- Created: 2015-12-23T00:04:59.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2015-12-24T08:57:02.000Z (over 10 years ago)
- Last Synced: 2025-04-23T21:32:09.432Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 549 KB
- Stars: 41
- Watchers: 7
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-android-performance - Probe - Android performance instrumentation tool, comparing application vital signs (CPU usage, memory utilization, number of threads, disk usage, etc.) across app versions, detecting performace regressions. (Articles / Network Profiling Tools)
README
# Probe
Android applications performance testing tool.
#Usage
There are two ways to use probe:
1. As a command line tool. This will run a basic test case of Restart ----> Steady state
```sh
# python console.py --package your.package.name --activity TheActivityToRunProbeOn --repeat-count x --timeout y
# python console.py --help
Options:
--package TEXT Package name to run Probe on
--activity TEXT Activity to restart within that package
--apk-path TEXT Optional: Path to installed APK, if you want APK analisys
--repeat-count INTEGER Times to repeat the test
--timeout INTEGER probe will stop when logcat output is silent for
that duration
--device-id TEXT device_id to send commands to
```
2. As a python module, as a part of your UI tests start probe at the begining of each test, and stop in when it ends (the following example shows how we did that with [Magneto](https://github.com/MagnetoTesting/magneto), setting up probe inside our `BaseTestCase`
```python
from probe.probe import Probe
class BaseTestCase(TestCase):
probe = None
def setup_method(self, method):
super(BaseTestCase, self).setup_method(method)
self.probe = Probe(package, activity, device_id)
probe.start()
def teardown_method(self, method):
super(BaseTestCase, self).teardown_method(method)
self.probe.stop()
```
##Background
In EverythingMe we used to have "performance blitzs" every now and then, when we (subjectivly) felt EverythingMe's launcher performance is not on par with what we would expect it to be. Then we would start manually measuring the app's vital signs using the following tools:
* **Memory utilization** - Dalvik, Native, PSS total:
```bash
adb shell dumpsys meminfo app.package.name
```
* **CPU utilization** - this one is tricky, you can certainly measure it with `top`, but in order to calculate "how much processing power was needed to complete the task you'd need to sum up the instantaneous CPU usage for the entire duration of the measurement. Instead, I chose to measure CPU ticks using data from proc stat
```bash
cat proc/{pid}/stat
```
* **Disk usage** - well, a simple `du` covers it
```bash
du -s /data/data/{package_name}
```
* **Thread count** - using `top` (#THR column)
* **a lot more** - with Android Studio's new(ish) tools, MAT and numerous logcat logs.
You are probably familiar with the huge amount of repetative work when doing such measurements and even though this is a tedious process, the collected results worth nothing without a comparison to previous measurements. This is where `probe` gets in the picture.
probe is desigend to measure these vital signs mentioned above (and much more), as a cli tool or as a python module.
##How does it work
Probe is designed to measure everything* automatically, and by everything I mean anything that has a logcat output or which can be queried using adb (dumpsys, dumpheap, profile, etc.) or a linux command (du, cat, top, etc.)
Each measurer registers to a certain registar, and the latter in turn triggers that measurer to do something.
There are three types of registrars:
##Registrars
Registars are metaclasses. when a class's metaclass is being set to be a registrar, it is automatically being registered to start/stop events and logcat output events.
**SnapshotRegistrar**: *more here soon*
**ContinuousRegistrar**: *more here soon*
**GlobalRegistrar**: *more here soon*
##Measurers
The actual code which collects the data is written here.
**Snapshot measurers**: When probe is triggered to stop measuring, it begins collecting data from all measurers registered to `SnapshotRegistrar`. It may get an instantaneous value of a measurer (current memory utilization, number of threads, CPU ticks, etc.)
**Continuous measurers**: When probe starts, it triggers an `adb logcat` instance. This instance output is being passed line by line to all measurers which registered to the `ContinuousRegistrar`. Each measurer filters relevant logcat lines using `is_matching(line)` function, and processes then in `process(line)`. The measurer handles its own state machine (whether its a simple line counter which only measures the amount of processed lines, or a more complicated one measuring the frequency of `GC_FOR_ALLOC` lines).
**Global measurers**: A combination of both, a Continuous measurer, receiving relevant logcat lines, processing them (either just counting occurrences of this logcat line, or triggering a small state machine in it). the registar then takes a snapshot of what it accumulated when probe triggers it to stop.
* **Writing new measurers**:
This is how a measurer looks like:
```python
class CpuTicksUser(object):
"""
Cpu usage - user space (in ticks)
"""
PROC_STAT_USER_TICKS_LOC = 13
__metaclass__ = SnapshotRegistrar
def __init__(self):
pass
def name(self):
"""
the key of this measurement in the output dict
"""
return 'cpu_ticks_user'
def value(self):
"""
the value of this measurement in the output dict
"""
pid = runtime.get_instance().get_pid()
procstat = adb.get_instance().shell('cat /proc/%s/stat' % pid)['stdout'][0].split()
return int(procstat[PROC_STAT_USER_TICKS_LOC])
```
## Querying probe's DB
Probe writes output to a provided DB (or a local sql lite (mydatabase.db) if no connection string is provided.
querying probe's db for `cpu_ticks` (both user and kernel ticks) values in last 100 builds:
```sql
SELECT p.Build_number, version_code, Cpu_ticks_user, Cpu_ticks_kernel FROM collector p
JOIN ((SELECT Build_number, max(Created_at) as max_time
FROM collector
GROUP BY Build_number)) m ON (p.Build_number=m.Build_number and p.created_at=m.max_time)
ORDER BY p.Build_number desc
LIMIT 100
```
And that's how it looks like in [redash](https://github.com/getredash/redash):
## Important note
**It is important to emphesize that probe is a benchmark to itself! meaning it needs to run consecutively on the same device in order to produce valid results (of course SGS4 would have a higher PSS total value than SGS3, it has 225% the pixle count of SGS3).**
##Some more graphs
or "this is what we measured on EverythingMe Launcher"
