{"id":34888477,"url":"https://github.com/wp-labs/warp-parse","last_synced_at":"2026-04-26T16:05:32.857Z","repository":{"id":328783992,"uuid":"1116685862","full_name":"wp-labs/warp-parse","owner":"wp-labs","description":"Focusing on building industry-leading ETL engines.","archived":false,"fork":false,"pushed_at":"2026-03-24T09:51:12.000Z","size":1761,"stargazers_count":166,"open_issues_count":38,"forks_count":9,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-03-24T14:38:31.155Z","etag":null,"topics":["data-collector","dataprocessing","etl","etl-framework","etl-pipeline","etl-process","events","logging","metrics","observability","oml","pipeline","siem","streaming","streaming-processing","telemetry","warp-parse","wp-parse","wpl"],"latest_commit_sha":null,"homepage":"https://warpparse.ai/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wp-labs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.en.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-12-15T08:31:48.000Z","updated_at":"2026-03-22T08:06:48.000Z","dependencies_parsed_at":"2026-01-05T00:10:08.816Z","dependency_job_id":"1036e0a6-a513-40c9-9f8f-7ae582ce75a1","html_url":"https://github.com/wp-labs/warp-parse","commit_stats":null,"previous_names":["wp-labs/warp-parse"],"tags_count":81,"template":false,"template_full_name":null,"purl":"pkg:github/wp-labs/warp-parse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wp-labs%2Fwarp-parse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wp-labs%2Fwarp-parse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wp-labs%2Fwarp-parse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wp-labs%2Fwarp-parse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wp-labs","download_url":"https://codeload.github.com/wp-labs/warp-parse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wp-labs%2Fwarp-parse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31291070,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-collector","dataprocessing","etl","etl-framework","etl-pipeline","etl-process","events","logging","metrics","observability","oml","pipeline","siem","streaming","streaming-processing","telemetry","warp-parse","wp-parse","wpl"],"created_at":"2025-12-26T04:24:54.420Z","updated_at":"2026-04-01T19:07:50.306Z","avatar_url":"https://github.com/wp-labs.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/WP—LOGO.V2.png\" alt=\"Warp Parse Logo\" width=\"200\"/\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eWarp Parse\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/wp-labs/warp-parse/actions/workflows/build-and-test.yml\"\u003e\n    \u003cimg src=\"https://github.com/wp-labs/warp-parse/actions/workflows/build-and-test.yml/badge.svg\" alt=\"Build \u0026 Test\"/\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/wp-labs/warp-parse/actions/workflows/release.yml\"\u003e\n    \u003cimg src=\"https://github.com/wp-labs/warp-parse/actions/workflows/release.yml/badge.svg\" alt=\"Release\"/\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.apache.org/licenses/LICENSE-2.0\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/License-Apache%202.0-blue.svg\" alt=\"License: Apache 2.0\"/\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n---\n\nWarp Parse is a high-performance Rust ETL engine built for observability, security, real-time risk control, and data platform teams. It focuses on log/telemetry ingestion, parsing, and transformation, providing extreme throughput parsing (WPL), transformation (OML), routing, unified connector APIs, and streamlined operations.\n\n## ✨ Core Features\n\n- **🚀 Extreme throughput:** EPS (Events Per Second) significantly surpasses Vector across multiple scenarios, with 2~6x performance advantages in core scenarios like fixed rate and large logs (see [Performance Report](https://github.com/wp-labs/wp-examples/tree/main/benchmark/report)).\n- **📝 Readable rules:** Self-developed WPL (Parse DSL) + OML (Transform DSL) offer far superior readability and maintainability compared to regular expressions and Lua scripts.\n- **🔌 Unified connectors:** Built on standardized `wp-connector-api` interface design, enabling community developers to rapidly extend multi-source log connector ecosystem.\n- **🛠️ Ops friendly:** Single binary deployment with full configuration-based management; includes `wproj`, `wpgen`, `wprescue` tool suite to reduce operational costs.\n- **🧠 Knowledge transformation:** Built-in in-memory database supports real-time SQL queries for log data field enrichment and correlation analysis.\n- **🎯 Data routing:** Flexible routing based on rule engine and transformation models, supporting multi-path data replication, precise filtering, and target distribution.\n\n## Performance\nWarpParse VS Vector、LogStash [Report](https://github.com/wp-labs/wp-examples/tree/main/benchmark/report)\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"images/wp-pk3.jpg\"  width=\"1200\"/\u003e\n\u003c/p\u003e\n\n## Setup\n\n```bash\n## 稳定版\ncurl -sSf https://get.warpparse.ai/inst-x.sh | bash -s -- wparse\n## 验证版\ncurl -sSf https://get.warpparse.ai/inst-x.sh | bash -s -- wparse beta\n## 开发版\ncurl -sSf https://get.warpparse.ai/inst-x.sh | bash -s -- wparse alpha\n## install wp-skills\ncurl -sSf https://get.warpparse.ai/inst-x.sh | bash -s -- wp-skills\n\n```\n\n## 🤝 Community \u0026 Collaboration\n\n### 1. Developer Contributions\nWe welcome all developers to participate in WarpParse development, whether it's feature development, bug fixes, or documentation improvements:\n- Contribution Guide: [CONTRIBUTING.md](CONTRIBUTING.md) (Includes PR submission process, code standards, and Issue feedback templates)\n- Issue Tracking: [GitHub Issues](https://github.com/wp-labs/warp-parse/issues)\n- Community Discussion: GitHub Discussions\n\n### 2. Enterprise/Vendor Partnerships\nIf your product is a **security threat detection platform, operations observability system, or cloud-native logging service** and requires high-performance log parsing capabilities:\n- You can directly integrate WarpParse open-source edition for free - we provide comprehensive technical documentation and integration guide support;\n- For customized adaptation, joint solution testing, or performance tuning, contact us via: coop@warpparse.ai\n\u003e Note: After integration, simply mention \"Built with WarpParse high-performance log parsing engine\" in your product's technical documentation - no additional authorization required.\n\n## 📄 License\n\n**WarpParse core engine and supporting toolchain (including WPL/OML parsers, wp-connector-api, tool suite, etc.) are licensed under Apache License 2.0.**\n\nYou are free to use, modify, and distribute the source code and derivative works of this project. When embedding into closed-source commercial products, you do not need to open-source your proprietary business code; when distributing modified derivative works, you must retain this license statement and copyright information.\n\nFor details, please refer to the [LICENSE](LICENSE) file in the repository root.\n\n---\n\n# Warp Parse（中文版）\n\n\u003cp align=\"center\"\u003e\n  \u003cstrong\u003e高性能 Rust ETL 引擎，专为极致日志处理而设计\u003c/strong\u003e\n\u003c/p\u003e\n\n---\n\n面向可观测性、安全、实时风控、数据平台团队的高性能 ETL 引擎，专注于日志/事件接入、解析与转换，提供高吞吐解析（WPL）、转换（OML）、路由、统一连接器 API 及极简运维体验。\n\n## ✨ 核心特性\n\n- **🚀 极致吞吐：** 多场景下 EPS（事件处理速率）全面超越 Vector，固定速率/大日志等核心场景性能优势达 2~6 倍（详见[性能测试报告](https://github.com/wp-labs/wp-examples/tree/main/benchmark/report)）。\n- **📝 规则易编写：** 自研 WPL（解析 DSL）+ OML（转换 DSL），可读性、可维护性远超正则表达式与 Lua 脚本。\n- **🔌 连接器统一：** 基于 `wp-connector-api` 标准化接口设计，支持社区开发者快速扩展多源日志连接器生态。\n- **🛠️ 运维友好：** 单二进制文件部署，全配置化管理；配套 `wproj`、`wpgen`、`wprescue` 工具套件，降低运维成本。\n- **🧠 知识转换：** 内置内存数据库支持 SQL 实时查询，实现日志数据字段富化与关联分析。\n- **🎯 数据路由：** 基于规则引擎与转换模型的灵活路由，支持多路数据复制、精准过滤与目标分发。\n\n## 🤝 社区与合作\n\n### 1. 开发者贡献\n我们欢迎所有开发者参与 WarpParse 的迭代，无论是功能开发、Bug 修复还是文档完善：\n- 贡献指南：[CONTRIBUTING.md](CONTRIBUTING.md)（内含 PR 提交流程、代码规范、Issue 反馈模板）\n- 问题反馈：[GitHub Issues](https://github.com/wp-labs/warp-parse/issues)\n- 交流社群：GitHub Discussions\n\n### 2. 企业/厂商合作\n如果你的产品是 **安全威胁检测平台、运维观测系统、云原生日志服务**，需要高性能日志解析能力：\n- 可直接免费集成 WarpParse 开源版，我们提供完整的技术文档与集成指南支持；\n- 如需定制化适配、联合方案测试、性能调优，可通过官方邮箱联系：coop@warpparse.ai\n\u003e 注：集成后只需在你的产品技术文档中注明「基于 WarpParse 高性能日志解析引擎构建」，无需额外申请授权。\n\n## 📄 许可协议\n\n**WarpParse 核心引擎及配套工具链（含 WPL/OML 解析器、wp-connector-api、工具套件等）均采用 Apache License 2.0 开源协议授权**。\n\n你可自由使用、修改、分发本项目源码及衍生作品，嵌入闭源商业产品时无需开源自有业务代码；分发修改后的衍生作品时，需保留本协议声明及版权信息。\n\n详情请参阅仓库根目录 [LICENSE](LICENSE) 文件。\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwp-labs%2Fwarp-parse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwp-labs%2Fwarp-parse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwp-labs%2Fwarp-parse/lists"}