{"id":43335052,"url":"https://github.com/trumank/ser-hex","last_synced_at":"2026-02-02T00:25:39.495Z","repository":{"id":219023296,"uuid":"747968294","full_name":"trumank/ser-hex","owner":"trumank","description":"Serialization tracing and visualization tools","archived":false,"fork":false,"pushed_at":"2025-06-12T04:03:05.000Z","size":339,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-12T05:23:28.459Z","etag":null,"topics":["instrumentation","reverse-engineering","serialization","tracing"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/trumank.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-01-25T01:54:51.000Z","updated_at":"2025-06-12T04:03:08.000Z","dependencies_parsed_at":"2024-01-25T02:59:08.382Z","dependency_job_id":"b4c1ae7d-6dc9-4c14-a705-e1ef60ee4926","html_url":"https://github.com/trumank/ser-hex","commit_stats":null,"previous_names":["trumank/ser-hex"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/trumank/ser-hex","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trumank%2Fser-hex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trumank%2Fser-hex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trumank%2Fser-hex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trumank%2Fser-hex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/trumank","download_url":"https://codeload.github.com/trumank/ser-hex/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/trumank%2Fser-hex/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28996637,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-01T23:10:54.274Z","status":"ssl_error","status_checked_at":"2026-02-01T23:10:47.298Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["instrumentation","reverse-engineering","serialization","tracing"],"created_at":"2026-02-02T00:25:38.595Z","updated_at":"2026-02-02T00:25:39.486Z","avatar_url":"https://github.com/trumank.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ser-hex\n\nSerialization tracing and visualization tools.\n\nAttempts to answer the question \"where did these bytes come from??\" when\nexamining opaque binary blobs of data.\n\n## ser-hex-tui\n\n```console\ncargo run --release --bin ser-hex-tui examples/bson/trace.json\n```\n\n![ser-hex-tui BSON trace](https://github.com/user-attachments/assets/4b48b982-9a30-44dd-b55c-c58ef9b86c73)\n\n\n## trace format\n\nThe trace output contains the binary data and a tree of stream actions (Read/Seek/Span).\n\n```json\n{\n  \"data\": \"DAAAAEhlbGxvIFdvcmxkIQ==\",\n  \"start_index\": 0,\n  \"root\": {\n    \"Span\": {\n      \"name\": \"pascal_string\",\n      \"actions\": [\n        {\n          \"Span\": {\n            \"name\": \"length\",\n            \"actions\": [\n              {\n                \"Read\": 4\n              }\n            ]\n          }\n        },\n        {\n          \"Read\": 12\n        }\n      ]\n    }\n  }\n}\n```\n\n## capturing a trace\n\nThere are two methods of capturing trace data from rust:\n\n### rust Tracing instrumentation\n\nBuilds spans by listening to `tracing::instrument`'d functions. This results\nin accurately nested spans but requires manual annotation of functions.\n\n```rust\nlet mut input = Cursor::new([1, 2, 3]);\nser_hex::read_incremental(\"trace.json\", \u0026mut input, read)?;\n\n#[tracing::instrument(skip_all)]\nfn read\u003cR: Read\u003e(input: \u0026mut R) -\u003e std::io::Result\u003c()\u003e {\n    input.read_exact(\u0026mut [0; 3])?;\n    Ok(())\n}\n```\n\n\n### backtrace captures\n\nThe second option is to construct spans from backtrace captures. This does not\nrequire sprinkling instrumentation annotations all over the serialization code,\nbut can result in lower quality trace data. Since backtraces are captured only\non read/seek events, it's impossible to know how far up the stack control flow\nwent between reads, which can lead to inaccurately reconstructed spans.\n\nIn theory, if stack frame push/pop events could be hooked at the hardware\nlevel or via [emulation](https://www.unicorn-engine.org/), they could be made\naccurate, but I have not explored this route yet. Still with this limitation, it\nprovides useful data with little effort.\n\n```rust\nlet mut input = Cursor::new([1, 2, 3]);\nlet mut tracer = ser_hex_tracer::TracerReader::new_options(\n    \u0026mut input,\n    ser_hex_tracer::TracerOptions { skip_frames: 3 }, // number of top level stack frames to omit from trace\n);\nlet res = read(\u0026mut tracer);\ntracer.trace().save(\"trace.json\").unwrap();\n\nfn read\u003cR: Read\u003e(input: \u0026mut R) -\u003e std::io::Result\u003c()\u003e {\n    input.read_exact(\u0026mut [0; 3])?;\n    Ok(())\n}\n```\n\n### tracing other streams or non-rust code\n\nIt is possible to trace arbitrary native code by hooking the necessary stream\nfunctions and calling the corresponding functions on your `Tracer` object. See\n[trace_factorio](examples/trace_factorio) and [trace_drg](examples/trace_drg)\nfor examples of tracing data streams implemented in C++.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrumank%2Fser-hex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftrumank%2Fser-hex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftrumank%2Fser-hex/lists"}