{"id":39132102,"url":"https://github.com/y-scope/log-surgeon","last_synced_at":"2026-01-17T21:19:02.485Z","repository":{"id":173138793,"uuid":"607748648","full_name":"y-scope/log-surgeon","owner":"y-scope","description":"A performant log parsing library","archived":false,"fork":false,"pushed_at":"2026-01-12T07:08:30.000Z","size":633,"stargazers_count":9,"open_issues_count":47,"forks_count":10,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-01-12T17:14:49.453Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/y-scope.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-02-28T15:47:26.000Z","updated_at":"2026-01-05T16:57:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"75ff22d0-9e84-4cf8-9eba-3931b367e74b","html_url":"https://github.com/y-scope/log-surgeon","commit_stats":null,"previous_names":["y-scope/log-surgeon"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/y-scope/log-surgeon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/y-scope%2Flog-surgeon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/y-scope%2Flog-surgeon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/y-scope%2Flog-surgeon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/y-scope%2Flog-surgeon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/y-scope","download_url":"https://codeload.github.com/y-scope/log-surgeon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/y-scope%2Flog-surgeon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28518618,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T18:55:29.170Z","status":"ssl_error","status_checked_at":"2026-01-17T18:55:03.375Z","response_time":85,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-17T21:19:02.420Z","updated_at":"2026-01-17T21:19:02.472Z","avatar_url":"https://github.com/y-scope.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# log-surgeon: A performant log parsing library\n\n[![CLP on Zulip](https://img.shields.io/badge/zulip-yscope--clp%20chat-1888FA?logo=zulip)](https://yscope-clp.zulipchat.com/)\n\n`log-surgeon` is a library for high-performance parsing of unstructured text\nlogs. It allows users to parse and extract information from the vast amount of\nunstructured logs generated by today's open-source software.\n\nSome of the library's features include:\n\n* Parsing and extracting variable values like the log event's log-level and any\n  other user-specified variables, no matter where they appear in each log event.\n* Parsing by using regular expressions for each variable type rather than\n  regular expressions for an entire log event.\n* Improved latency, and memory efficiency compared to popular regex engines.\n* Parsing multi-line log events (delimited by timestamps).\n\nNote that `log-surgeon` is *not* a generic regex engine and does impose [some\nconstraints](docs/parsing-constraints.md) on how log events can be parsed.\n\n## Motivating example\n\nLet's say we want to parse and inspect multi-line log events like this:\n\n```\n2023-02-23T18:10:14-0500 DEBUG task_123 crashed. Dumping stacktrace:\n#0  0x000000000040110e in bar () at example.cpp:6\n#1  0x000000000040111d in bar () at example.cpp:10\n#2  0x0000000000401129 in main () at example.cpp:15\n```\n\nUsing the [example schema file](examples/schema.txt) which includes these rules:\n\n```\ntimestamp:\\d{4}\\-\\d{2}\\-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\-\\d{4}\n...\nloglevel:INFO|DEBUG|WARN|ERROR\n```\n\nWe can parse and inspect the events as follows:\n\n```cpp\n// Define a reader to read from your data source\nReader reader{/* \u003cOmitted\u003e */};\n\n// Instantiate the parser\nReaderParser parser{\"examples/schema.txt\"};\nparser.reset_and_set_reader(reader);\n\n// Get the loglevel variable's ID\noptional\u003cuint32_t\u003e loglevel_id{parser.get_variable_id(\"loglevel\")};\n// \u003cOmitted validation of loglevel_id\u003e\n\nwhile (false == parser.done()) {\n    if (ErrorCode err{parser.parse_next_event()}; ErrorCode::Success != err) {\n        throw runtime_error(\"Parsing Failed\");\n    }\n    LogEventView const\u0026 event{parser.get_log_parser().get_log_event_view()};\n\n    // Get and print the timestamp\n    Token* timestamp{event.get_timestamp()};\n    if (nullptr != timestamp) {\n        cout \u003c\u003c \"timestamp: \" \u003c\u003c timestamp-\u003eto_string_view() \u003c\u003c endl;\n    }\n\n    // Get and print the log-level\n    auto const\u0026 loglevels{event.get_variables(*loglevel_id)};\n    if (false == loglevels.empty()) {\n        // In case there are multiple matches, just get the first one\n        cout \u003c\u003c \"loglevel:\" \u003c\u003c loglevels[0]-\u003eto_string_view() \u003c\u003c endl;\n    }\n\n    // Other analysis...\n\n    // Print the entire event\n    cout \u003c\u003c event.to_string() \u003c\u003c endl;\n}\n```\n\nFor advanced uses, `log-surgeon` also has a\n[BufferParser](examples/buffer-parser.cpp) that reads directly from a buffer.\n\n## Building and installing\n\nRequirements:\n\n* CMake \u003e= 3.22.1\n* GCC \u003e= 10 or Clang \u003e= 7\n* [Catch2] \u003e= 3.8.1\n* [fmt] \u003e= 11.2.0\n* [GSL] \u003e= 4.0.0\n* [Task] \u003e= 3.38\n* [uv] \u003e= 0.7.10\n* [ystdlib-cpp] \u003e= 0.1.0\n\nTo build and install the project to `$HOME/.local`:\n\n```shell\ntask log-surgeon:install-release INSTALL_PREFIX=\"$HOME/.local\"\n```\n\nOr to only build the project:\n\n```shell\ntask log-surgeon:build-release\n```\n\nTo build the debug version:\n\n```shell\ntask log-surgeon:build-debug\n```\n\n## Examples\n\n[examples](examples) contains programs demonstrating usage of the library. See\n[examples/README.md](examples/README.md) for information on building and running the examples.\n\n## Documentation\n\n* [docs](docs) contains more detailed documentation including:\n  * The [schema specification](docs/schema.md), which describes the syntax for\n    writing your own schema\n  * `log-surgeon`'s [design objectives](docs/design-objectives.md)\n\n### Documentation site\n\nThe project includes a documentation site that's useful for exploring functionality and test\ncoverage. In particular, it documents all unit tests, with additional detail for API-level tests.\n\nTo generate and view the files:\n\n* Run `task docs:site`.\n* Open `build/docs/html/index.html` in your preferred browser.\n\nTo host the site locally and view it:\n\n* Run `task docs:serve`.\n* Open the URL output by the task in your preferred browser.\n\n## Testing\n\nTo build and run all unit tests:\n\n```shell\ntask test:run-debug\n```\n\nWhen generating targets, the CMake variable `BUILD_TESTING` is followed (unless overruled by setting\n`log_surgeon_BUILD_TESTING` to false). By default, if built as a top-level project, `BUILD_TESTING`\nis set to true and unit tests are built.\n\n## Linting\n\nBefore submitting a PR, ensure you've run our linting tools and either fixed any violations or\nsuppressed the warning.\n\n### Running the linters\n\nTo report all errors, run:\n\n```shell\ntask lint:check\n```\n\nTo automatically fix any supported format or linting errors, run:\n\n```shell\ntask lint:fix\n```\n\n## Providing feedback\n\nYou can use GitHub issues to [report a bug][bug-report] or [request a feature][feature-req].\n\nJoin us on [Zulip](https://yscope-clp.zulipchat.com/) to chat with developers\nand other community members.\n\n## Known issues\n\nThe following are issues we're aware of and working on:\n* Schema rules must use ASCII characters. We will release UTF-8 support in a\n  future release.\n* Timestamps must appear at the start of the message to be handled specially\n  (than other variable values) and support multi-line log events.\n* A variable pattern has no way to match text around a variable, without having\n  it also be a part of the variable.\n  * Support for submatch extraction will be coming in a future release.\n\n[bug-report]: https://github.com/y-scope/log-surgeon/issues/new?assignees=\u0026labels=bug\u0026template=bug-report.yaml\n[Catch2]: https://github.com/catchorg/Catch2/tree/devel\n[feature-req]: https://github.com/y-scope/log-surgeon/issues/new?assignees=\u0026labels=enhancement\u0026template=feature-request.yaml\n[fmt]: https://github.com/fmtlib/fmt\n[GSL]: https://github.com/microsoft/GSL\n[Task]: https://taskfile.dev/\n[uv]: https://docs.astral.sh/uv\n[ystdlib-cpp]: https://github.com/y-scope/ystdlib-cpp\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fy-scope%2Flog-surgeon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fy-scope%2Flog-surgeon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fy-scope%2Flog-surgeon/lists"}