{"id":47931700,"url":"https://github.com/jonhoo/avdl","last_synced_at":"2026-04-04T07:18:18.812Z","repository":{"id":336920702,"uuid":"1151675817","full_name":"jonhoo/avdl","owner":"jonhoo","description":"Rust port of avro-tools' IDL tooling","archived":false,"fork":false,"pushed_at":"2026-03-14T17:03:27.000Z","size":1377,"stargazers_count":30,"open_issues_count":4,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-15T03:39:22.361Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonhoo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-06T19:00:08.000Z","updated_at":"2026-03-14T17:03:23.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/jonhoo/avdl","commit_stats":null,"previous_names":["jonhoo/avdl"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/jonhoo/avdl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Favdl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Favdl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Favdl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Favdl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonhoo","download_url":"https://codeload.github.com/jonhoo/avdl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Favdl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31391214,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T04:26:24.776Z","status":"ssl_error","status_checked_at":"2026-04-04T04:23:34.147Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-04T07:18:17.542Z","updated_at":"2026-04-04T07:18:18.797Z","avatar_url":"https://github.com/jonhoo.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# avdl\n\n[![codecov](https://codecov.io/gh/jonhoo/avdl/graph/badge.svg?token=WP4ZHFDYVO)](https://codecov.io/gh/jonhoo/avdl)\n\nA Rust port of Apache Avro's `avro-tools idl` and `idl2schemata`\ncommands. Compiles [Avro IDL](https://avro.apache.org/docs/1.12.0/idl-language/)\n(`.avdl`) files to Avro Protocol JSON (`.avpr`) or Schema JSON (`.avsc`).\n\n## Usage\n\n```sh\n# .avdl → protocol JSON\navdl idl input.avdl output.avpr\n\n# .avdl → individual schema files\navdl idl2schemata input.avdl outdir/\n\n# stdin/stdout\navdl idl \u003c input.avdl\n\n# additional import search paths\navdl idl --import-dir ./extra/ input.avdl\n```\n\n## Install\n\n```sh\ncargo install avdl\n```\n\nOr build and run from a local checkout:\n\n```sh\ncargo build\ncargo run -- idl input.avdl\n```\n\n## Testing\n\n```sh\ncargo test\n```\n\nThe integration tests parse the same `.avdl` input files shipped in\nthe [Apache Avro](https://github.com/apache/avro/) test suite and\ncompare output semantically against the expected `.avpr`/`.avsc`\nreference files. All test cases pass, alongside unit tests for\nserialization, error reporting, and edge cases. The tool is functional\nbut young — bug reports are welcome.\n\n## Intentional divergences from Java\n\nThis tool aims for semantic correctness against the Java `avro-tools`\noutput, but deliberately differs in a few ways:\n\n- **JSON formatting is not byte-identical.** Whitespace and array\n  line-breaking style may differ. JSON object keys are sorted\n  alphabetically, whereas Java avro-tools preserves insertion order.\n  The output parses to the same logical structure.\n\n- **Import search paths replace Java classpath.** avro-tools resolves\n  `import` paths via the JVM classpath; this tool uses explicit\n  `--import-dir` flags instead, which serve the same purpose without\n  requiring a JVM.\n\n- **Better error diagnostics.** Errors include source context with the\n  offending token underlined, powered by [miette](https://docs.rs/miette).\n  The Rust tool also gives clear error messages in cases where\n  avro-tools 1.12.1 crashes with unchecked exceptions (e.g.,\n  `NoSuchElementException` for duplicate message parameters,\n  `NullPointerException` for reserved type names via backtick escapes).\n\n  For example, given a record with a duplicate field name:\n\n  ```avro\n  record User {\n      string name;\n      int name;\n  }\n  ```\n\n  **avro-tools** produces a Java stack trace:\n\n  ```\n  Exception in thread \"main\" org.apache.avro.SchemaParseException:\n    org.apache.avro.AvroRuntimeException: Field already used: name type:STRING pos:0\n      at org.apache.avro.idl.IdlReader.parse(IdlReader.java:220)\n      ... 5 more\n  ```\n\n  **avdl** points directly at the problem:\n\n  ```\n  Error:   × parse IDL source\n    ├─▶ parse `dup-field.avdl`\n    ╰─▶ duplicate field 'name' in record 'User'\n     ╭─[dup-field.avdl:4:13]\n   3 │         string name;\n   4 │         int name;\n     ·             ──┬─\n     ·               ╰── duplicate field 'name' in record 'User'\n   5 │     }\n     ╰────\n  ```\n\n- **Nested unions are rejected per spec.** The Avro specification\n  states \"Unions may not immediately contain other unions.\" This tool\n  enforces this rule, while Java avro-tools 1.12.1 incorrectly accepts\n  nested unions and silently produces an empty union `[]` in the JSON\n  output. If you have `.avdl` files that compiled with Java but fail\n  here, the nested union is a spec violation that Java should have\n  caught.\n\n- **Float formatting uses serde_json defaults.** Java renders large\n  or small float/double values in scientific notation (e.g.,\n  `-1.0E12`); this tool uses `serde_json`'s decimal representation\n  (e.g., `-1000000000000.0`). Both parse to the same numeric value.\n\n- **Namespace validation covers all segments.** Rust validates every\n  dot-separated segment of namespace names. Java's\n  `IdlReader.namespace()` loop skips the last segment, so a namespace\n  like `org.valid.0bad` passes Java's validation but is correctly\n  rejected by the Rust tool.\n\n- **Faster execution.** As a native binary, avdl avoids JVM startup\n  overhead. On real-world `.avdl` files it completes in single-digit\n  milliseconds — roughly 50× faster than `avro-tools idl` for\n  typical inputs, narrowing to ~6× on unusually large (1 MB) files\n  where JVM startup is a smaller fraction of the total.\n\n## Attribution\n\nThis is a \"powercoded\" port of the Java Avro tools — meaning it was\nbuilt with LLM assistance, incremental human guidance and review, and\nstrong testing against the upstream test suite.\n\nAll credit for Avro itself, the original tooling, and the underlying\ntest suite belongs to the [Apache Avro](https://github.com/apache/avro/)\nproject.\n\nKey upstream sources this port is based on:\n\n- [IdlTool.java](https://github.com/apache/avro/blob/c499eefb48aa2db906c7bca14a047223806f36db/lang/java/tools/src/main/java/org/apache/avro/tool/IdlTool.java) — the Java `avro-tools idl` entry point\n- [IdlReader.java](https://github.com/apache/avro/blob/c499eefb48aa2db906c7bca14a047223806f36db/lang/java/idl/src/main/java/org/apache/avro/idl/IdlReader.java) — the Java IDL transformer\n- [Idl.g4](https://github.com/apache/avro/blob/c499eefb48aa2db906c7bca14a047223806f36db/share/idl_grammar/org/apache/avro/idl/Idl.g4) — the ANTLR grammar for Avro IDL\n\n## Background\n\nThe initial version of this crate was built during a [live stream on YouTube](https://youtu.be/vmKvw73V394).\nThe live stream ends at commit [`fac28dd`](https://github.com/jonhoo/avdl/commit/fac28dd).\nPost-stream development is documented in [post-stream-summary.md](post-stream-summary.md).\n\n## License\n\nLicensed under the [Apache License, Version 2.0](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonhoo%2Favdl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonhoo%2Favdl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonhoo%2Favdl/lists"}