https://github.com/openclaw/openclaw-rtt
RTT timing measurements across OpenClaw npm releases.
https://github.com/openclaw/openclaw-rtt
Last synced: 29 days ago
JSON representation
RTT timing measurements across OpenClaw npm releases.
- Host: GitHub
- URL: https://github.com/openclaw/openclaw-rtt
- Owner: openclaw
- Created: 2026-05-01T08:29:22.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2026-05-15T15:33:49.000Z (30 days ago)
- Last Synced: 2026-05-15T16:11:10.875Z (30 days ago)
- Language: JavaScript
- Size: 123 KB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# OpenClaw RTT
Time-series store for OpenClaw Telegram and Discord RTT measurements.
The measurement harness lives in `openclaw/openclaw`. This repo stores normalized results and the README dashboard.
## Import A Run
From this repo:
```sh
node scripts/import-result.mjs ../clawdbot/runs//result.json
node scripts/import-discord-rtt.mjs samples.tsv --spec openclaw@main --version
node scripts/summary.mjs
```
Expected source shape is the `result.json` emitted by:
```sh
pnpm rtt openclaw@beta
pnpm rtt openclaw@beta --samples 20
pnpm rtt openclaw@latest
pnpm rtt openclaw@2026.4.30 --provider live-frontier
```
## Data
- `data/rtt.jsonl`: append-only graph source, one normalized run per line.
- `runs//result.json`: copied per-run result record for audit/debug.
- `data/discord-rtt.jsonl`: append-only Discord RTT graph source.
- `discord-runs//result.json`: copied per-run Discord RTT record.
Raw Telegram QA artifacts stay in the OpenClaw repo artifact directory unless explicitly copied in later.
## Dashboard
Current `openclaw@main` snapshot. Telegram and Discord run on separate schedules, so each row keeps its own update time.
Latest imported main run: `2026-05-16T07:18:04.544Z`
| Transport | Version/ref | Result | Samples | p50 | p95 | Updated |
|---|---:|---:|---:|---:|---:|---:|
| Telegram | `2026.5.16+df0d061c7a` | Pass | 20 | `1,126ms` | `2,089ms` | `2026-05-16T06:56:44.998Z` |
| Discord | `2026.5.16+e975c3b212` | Pass | 20 | `6,590ms` | `6,952ms` | `2026-05-16T07:18:04.544Z` |
## Telegram Release Sweep
Measured with the OpenClaw repo black-box harness on Blacksmith using `mock-openai`, scenario `telegram-mentioned-message-reply`, 20 target normal-reply samples, 240s canary timeout, and 30s per-sample timeout.
The SUT is the published package running its own Telegram bot. The OpenClaw repo only supplies the mock model server and Telegram driver. `p50` is the median normal-reply RTT. Log notes: [2026-05-02 Testbox stable sweep](logs/2026-05-02-testbox-stable-sweep.md).
End-to-end latency measured over Telegram using `mock-openai`.
| npm version | Result | Samples | p50 | p95 | Updated |
|---|---:|---:|---:|---:|---:|
| `2026.5.16-beta.1` | Pass | 20 | `1,196ms` | `1,969ms` | `2026-05-16T07:03:30.628Z` |
| `2026.5.14-beta.2` | Pass | 20 | `990ms` | `1,745ms` | `2026-05-15T11:25:24.501Z` |
| `2026.5.14-beta.1` | Pass | 20 | `1,004ms` | `3,915ms` | `2026-05-15T12:12:25.166Z` |
| `2026.5.12` | Pass | 20 | `2,858ms` | `23,061ms` | `2026-05-15T14:57:36.884Z` |
| `2026.5.9-beta.1` | Pass | 20 | `2,517ms` | `14,692ms` | `2026-05-09T13:46:04.963Z` |
| `2026.5.7` | Pass | 20 | `3,499ms` | `21,847ms` | `2026-05-07T22:34:10.660Z` |
| `2026.5.6` | Pass | 20 | `3,497ms` | `16,762ms` | `2026-05-07T01:10:20.034Z` |
| `2026.5.4` | Pass | 20 | `3,504ms` | `16,673ms` | `2026-05-05T12:55:16.706Z` |
| `2026.5.3` | Pass | 20 | `3,505ms` | `16,741ms` | `2026-05-04T09:49:15.970Z` |
| `2026.5.2` | Pass | 20 | `3,501ms` | `16,759ms` | `2026-05-03T01:14:23.528Z` |
| `2026.4.29` | Pass | 20 | `17,936ms` | `24,517ms` | `2026-05-02T06:17:26.914Z` |
| `2026.4.27` | Pass | 20 | `4,639ms` | `13,664ms` | `2026-05-02T06:11:35.226Z` |
| `2026.4.26` | Pass | 20 | `5,880ms` | `18,610ms` | `2026-05-02T06:05:19.893Z` |
| `2026.4.25` | Pass | 20 | `7,508ms` | `27,982ms` | `2026-05-02T05:59:20.038Z` |
| `2026.4.24` | Pass | 20 | `2,679ms` | `13,451ms` | `2026-05-02T05:54:14.272Z` |
| `2026.4.23` | Pass | 20 | `2,507ms` | `14,671ms` | `2026-05-02T05:49:14.713Z` |
| `2026.4.22` | Pass | 20 | `2,497ms` | `14,847ms` | `2026-05-02T05:34:51.113Z` |
| `2026.4.21` | Pass | 20 | `3,502ms` | `16,828ms` | `2026-05-02T05:43:01.319Z` |
| `2026.4.20` | Pass | 20 | `3,504ms` | `16,796ms` | `2026-05-02T05:26:57.320Z` |
| `2026.4.15` | Pass | 20 | `3,503ms` | `16,809ms` | `2026-05-02T05:20:28.620Z` |
## Discord Release Sweep
Measured with the OpenClaw Discord QA harness using `mock-openai`, scenario `discord-canary`, and Convex-managed CI credentials.
No release RTT runs have been imported yet.