{"id":43497041,"url":"https://github.com/varpulis/varpulis","last_synced_at":"2026-04-02T14:28:22.790Z","repository":{"id":334130478,"uuid":"1140181420","full_name":"varpulis/varpulis","owner":"varpulis","description":"Open source complex event processing solution in Rust","archived":false,"fork":false,"pushed_at":"2026-03-26T12:06:37.000Z","size":13064,"stargazers_count":2,"open_issues_count":6,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-26T13:39:27.922Z","etag":null,"topics":["cep","complex-event-processing","kubernetes","raft","rust","streaming-analytics","zdd"],"latest_commit_sha":null,"homepage":"https://www.varpulis-cep.com/landing","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/varpulis.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-22T23:46:20.000Z","updated_at":"2026-03-26T12:06:41.000Z","dependencies_parsed_at":"2026-02-03T11:02:52.480Z","dependency_job_id":null,"html_url":"https://github.com/varpulis/varpulis","commit_stats":null,"previous_names":["varpulis/varpulis"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/varpulis/varpulis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varpulis%2Fvarpulis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varpulis%2Fvarpulis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varpulis%2Fvarpulis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varpulis%2Fvarpulis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/varpulis","download_url":"https://codeload.github.com/varpulis/varpulis/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varpulis%2Fvarpulis/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31308079,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-02T12:59:32.332Z","status":"ssl_error","status_checked_at":"2026-04-02T12:54:48.875Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cep","complex-event-processing","kubernetes","raft","rust","streaming-analytics","zdd"],"created_at":"2026-02-03T11:01:08.036Z","updated_at":"2026-04-02T14:28:22.783Z","avatar_url":"https://github.com/varpulis.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/assets/logo.png\" width=\"320\" alt=\"Varpulis\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cstrong\u003eDetect kill chains your SIEM misses.\u003c/strong\u003e\u003cbr\u003eBehavioral sequence detection in Rust. Dual red/blue mode.\u003c/p\u003e\n\n[![CI](https://github.com/varpulis/varpulis/actions/workflows/ci.yml/badge.svg)](https://github.com/varpulis/varpulis/actions/workflows/ci.yml)\n[![crates.io](https://img.shields.io/crates/v/varpulis-cli.svg)](https://crates.io/crates/varpulis-cli)\n[![docs.rs](https://docs.rs/varpulis-core/badge.svg)](https://docs.rs/varpulis-core)\n[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue)](LICENSE-MIT)\n\n[Documentation](https://www.varpulis-cep.com/docs/) · [Live Demo](https://demo.varpulis-cep.com) · [Quick Start](#quick-start) · [Examples](examples/) · [Benchmarks](docs/PERFORMANCE_ANALYSIS.md)\n\n---\n\n```python\nstream FraudAlert = Events\n    .where(type == \"login\") as e1\n    -\u003e Events.where(type == \"transfer\") as e2\n    -\u003e Events.where(type == \"transfer\") as e3\n    .within(5m)\n    .where(e2.amount + e3.amount \u003e 10000)\n    .forecast(confidence: 0.8, horizon: 2m)\n    .alert(webhook: \"https://ops.example.com/fraud\", message: \"User {e1.user}: ${e2.amount + e3.amount}\")\n    .emit(user: e1.user, total: e2.amount + e3.amount)\n```\n\nLogin → two transfers over $10K within 5 minutes. `.forecast()` fires **before** the pattern completes. No other open-source CEP engine does this.\n\n## Quick Start\n\n```bash\ncurl -sSf https://raw.githubusercontent.com/varpulis/varpulis/main/scripts/install.sh | sh\nvarpulis interactive --no-tui\n```\n\n```\nvpl\u003e event Tick: price: float\nvpl\u003e stream Spike = Tick .where(price \u003e 100) .emit(alert: \"spike\", price: price)\nvpl\u003e Tick { price: 42.0 }\nvpl\u003e Tick { price: 150.0 }\n→ Spike: {\"alert\":\"spike\",\"price\":150}\nvpl\u003e :save spike_detector.vpl\n```\n\nCopy-paste. 30 seconds. No files, no connectors, no Docker.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/assets/recordings/tui-split-pane.gif\" alt=\"Varpulis TUI\" width=\"720\"\u003e\n\u003c/p\u003e\n\nThe default `varpulis interactive` opens a split-pane TUI with topology, live events, input, and metrics. Add `--no-tui` for a plain text shell, `--json` for agent automation.\n\n## Security: Kill Chain Detection\n\nVarpulis detects **multi-step attack sequences** that single-event SIEM rules miss. Renamed PsExec? Different C2 tool? Doesn't matter — behavioral patterns catch what signature rules can't.\n\n```bash\n# Blue mode: detect kill chains in Sysmon logs\nvarpulis detect --rules rules/ --events sysmon.jsonl\n\n# Red mode: test which rules survive evasion\nvarpulis analyze --rules rules/ --baseline normal.jsonl --evasion evasion.jsonl\n```\n\n```\n┌───────────────────┬─────────────────────┬────────────┬────────────┬───────────┐\n│ Rule              ┆ MITRE               ┆ Baseline   ┆ Evasion    ┆ Verdict   │\n╞═══════════════════╪═════════════════════╪════════════╪════════════╪═══════════╡\n│ sigma_psexec      ┆ T1021.002           ┆ DETECT (1) ┆ MISS       ┆ EVADABLE  │\n├╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┤\n│ behavioral_psexec ┆ T1021.002,T1036.003 ┆ DETECT (1) ┆ DETECT (1) ┆ RESILIENT │\n└───────────────────┴─────────────────────┴────────────┴────────────┴───────────┘\n```\n\nValidated against real [MORDOR APT29](https://securitydatasets.com/) datasets at 25K+ events/sec. See the [SIEM Evasion Lab](docs/siem-evasion-lab-01-psexec.md) series for deep-dives on Sigma blind spots.\n\n## Why Varpulis?\n\n| | Varpulis | Flink CEP | Esper | Siddhi |\n|---|---|---|---|---|\n| **Temporal patterns** (Kleene `+/*`, negation, within) | Native (SASE+) | Limited | Yes | Partial |\n| **Predictive forecasting** | `.forecast()` built-in | No | No | No |\n| **Deployment** | Single binary (15 MB) | JVM cluster | Embedded JVM | Embedded JVM |\n| **DSL** | VPL (dedicated) | Java API | EPL | SiddhiQL |\n| **Throughput** | 1.5M evt/s (single core) | ~500K evt/s¹ | ~1M evt/s¹ | ~300K evt/s¹ |\n\n¹ Approximate figures from published benchmarks and vendor documentation; workload-dependent.\n\n**`.forecast()` is unique.** It uses Probabilistic Suffix Trees to predict that a pattern is *about to* complete — before the final event arrives. Combined with Hawkes process intensity estimation and conformal prediction intervals, it turns reactive CEP into proactive alerting.\n\n## Performance\n\n| What | Speed |\n|------|-------|\n| Core SASE+ pattern matching | **1.5M evt/s** |\n| Full VPL pipeline (filter + emit) | **410K evt/s** |\n| CLI end-to-end (file → process → output) | **256K evt/s** |\n| Multi-query Hamlet (50 concurrent) | **950K evt/s** |\n| Single-symbol prediction | **51 ns** |\n\nSingle core. [Detailed benchmarks →](docs/PERFORMANCE_ANALYSIS.md)\n\n## Connectors\n\n| | Status | Direction |\n|---|---|---|\n| MQTT, Kafka, NATS, HTTP | **Battle-tested** | In/Out |\n| PostgreSQL/MySQL/SQLite, Redis | Tested | In/Out |\n| Kinesis, S3, Elasticsearch, Pulsar, CDC | Available | Varies |\n\nEach connector is an independent crate. The default binary includes all; build with `--features mqtt,kafka` for a minimal binary.\n\n## Features\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eLanguage\u003c/strong\u003e\u003c/summary\u003e\n\n- Pipeline operators: `.where()`, `.window()`, `.aggregate()`, `.emit()`, `.to()`, `.alert()`\n- SASE+ patterns: sequences (`-\u003e`), Kleene closures (`+`, `*`), negation (`AND NOT`)\n- Forecasting: `.forecast()` — PST-based prediction with confidence and horizon\n- Alert webhooks: `.alert(webhook: \"url\", message: \"{field}\")` — fire-and-forget\n- Windows: tumbling, sliding, session, count-based\n- Aggregations: 15+ functions (sum, avg, ema, percentile, stddev, ...) — SIMD-accelerated\n- Joins: inner, LEFT, RIGHT, FULL outer with null-fill\n- Imperative: `var`, `if/else`, `while`, `for`, functions, lambdas\n- Compile-time meta-programming: `for row in 0..4:` generates streams\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eDeveloper Experience\u003c/strong\u003e\u003c/summary\u003e\n\n- Interactive TUI with split-pane topology/events/metrics (`varpulis interactive`)\n- Schema inference from sample data (`varpulis infer --input data.jsonl`)\n- Pipeline trace / explain mode (`--trace`)\n- Watch mode with auto-reload (`--watch`)\n- VS Code extension (LSP: diagnostics, completion, hover, go-to-definition)\n- MCP server for AI-assisted development\n- JSON-line protocol for agent automation (`--json`)\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003e\u003cstrong\u003eOperations\u003c/strong\u003e\u003c/summary\u003e\n\n- Single binary, Docker, Kubernetes (Helm chart included)\n- Coordinator/worker cluster with Raft consensus\n- Multi-tenant SaaS mode with RBAC and SSO/OIDC\n- Prometheus metrics, OpenTelemetry tracing, Grafana dashboards\n- RocksDB state persistence with optional AES-256-GCM encryption\n- Circuit breaker, dead letter queue, backpressure signaling\n\u003c/details\u003e\n\n## Documentation\n\n| | |\n|---|---|\n| [Getting Started](docs/tutorials/getting-started.md) | [Interactive Shell Tutorial](docs/tutorials/interactive-shell-tutorial.md) |\n| [VPL Language Tutorial](docs/tutorials/language-tutorial.md) | [SASE+ Patterns Guide](docs/guides/sase-patterns.md) |\n| [Forecasting Architecture](docs/architecture/forecasting.md) | [CLI Reference](docs/reference/cli-reference.md) |\n| [Cluster Tutorial](docs/tutorials/cluster-tutorial.md) | [Production Deployment](docs/PRODUCTION_DEPLOYMENT.md) |\n| [System Architecture](docs/architecture/system.md) | [All Tutorials →](docs/tutorials/) |\n\n## Contributing\n\nContributions welcome — see [CONTRIBUTING.md](CONTRIBUTING.md).\n\n## License\n\nDual-licensed under [MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE).\n\n## Acknowledgments\n\nSASE/SASE+ — [Wu et al. SIGMOD 2006](https://dl.acm.org/doi/abs/10.1145/1142473.1142520), [Agrawal et al. SIGMOD 2008](https://www.lix.polytechnique.fr/~yanlei.diao/publications/sase-sigmod08-long.pdf) · Hamlet — [Poppe et al. SIGMOD 2021](https://arxiv.org/abs/2101.00361) · Built with [Pest](https://pest.rs/) and [Tower-LSP](https://github.com/ebkalderon/tower-lsp)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvarpulis%2Fvarpulis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvarpulis%2Fvarpulis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvarpulis%2Fvarpulis/lists"}